| Message ID | 20191028212224.25941-1-dwilder@us.ibm.com |
|---|---|
| State | Superseded |
| Headers | show |
| Series | [ovs-dev] travis: support ppc64le builds | expand |
On 28.10.2019 22:22, David Wilder wrote: > Add support for travis-ci ppc64le builds. > > - Updated matrix in .travis.yml to include a ppc64le build. > - Added support to install packages needed by specific architectures. > > To keep the total build time at an acceptable level only a single build > job is included in the matrix for ppc64le. > > A build report example can be found here [1] > [0] http://travis-ci.org/ > [1] https://travis-ci.org/djlwilder/ovs/builds/604098141 > > Signed-off-by: David Wilder <dwilder@us.ibm.com> Hi David, Thanks for working on this. I have a couple of question regarding ppc64le support by TravisCI. It seems that they are not supporting this architecture officially and refusing[1] to solve any issues that appears while using it. There also no official documentation. It's kind of a hidden feature that some projects are using for their own risk [2]. Do you know why this happens or maybe you have some insights about what is going on/how it works? The API is also a bit strange because Travis started to officially support arm builds, but this is done via 'arch' knob, not the 'os'. Will it be changed over time for ppc64le? [1] https://travis-ci.community/t/ppc64le-arch-support-on-travis-ci-com-vs-travis-ci-org/2898/2 [2] https://github.com/openssl/openssl/commit/13da3ad00c80e1da816ca27f6c15b0ecee1bb0b8 Few code comments inline. > --- > .travis.yml | 5 +++-- > .travis/linux-prepare.sh | 18 ++++++++++++++---- > 2 files changed, 17 insertions(+), 6 deletions(-) > > diff --git a/.travis.yml b/.travis.yml > index 5676d9748..c99f26815 100644 > --- a/.travis.yml > +++ b/.travis.yml > @@ -14,7 +14,6 @@ addons: > apt: > packages: > - bc > - - gcc-multilib > - libssl-dev > - llvm-dev > - libjemalloc1 > @@ -24,7 +23,6 @@ addons: > - libelf-dev > - selinux-policy-dev > - libunbound-dev > - - libunbound-dev:i386 > - libunwind-dev > > before_install: ./.travis/${TRAVIS_OS_NAME}-prepare.sh > @@ -50,6 +48,9 @@ matrix: > - os: osx > compiler: clang > env: OPTS="--disable-ssl" > + - os: linux-ppc64le > + compiler: gcc > + env: OPTS="--disable-ssl" > > script: ./.travis/${TRAVIS_OS_NAME}-build.sh $OPTS > > diff --git a/.travis/linux-prepare.sh b/.travis/linux-prepare.sh > index e546d32cb..f3a9a6d44 100755 > --- a/.travis/linux-prepare.sh > +++ b/.travis/linux-prepare.sh > @@ -15,8 +15,18 @@ cd .. > pip install --disable-pip-version-check --user six flake8 hacking > pip install --user --upgrade docutils > > -if [ "$M32" ]; then > - # 32-bit and 64-bit libunwind can not be installed at the same time. > - # This will remove the 64-bit libunwind and install 32-bit version. > - sudo apt-get install -y libunwind-dev:i386 > +# Include packages needed by specific architectures. > +if [ $TRAVIS_ARCH == amd64 ]; then > + sudo apt-get install -y \ > + libunbound-dev:i386 \ > + gcc-multilib These packages are only needed for 32bit build, so you may just move them to the command that installs 32bit version of libunwind. And since you're not building 32bit with ppc64le, the code could look like: if [ "$M32" ]; then # install 32 bit libs fi if [ $TRAVIS_ARCH == ppc64le ]; then # install ppc64le specific things. fi > + > + if [ "$M32" ]; then > + # 32-bit and 64-bit libunwind can not be installed at the same time. > + # This will remove the 64-bit libunwind and install 32-bit version. > + sudo apt-get install -y libunwind-dev:i386 > + fi > + > +elif [ $TRAVIS_ARCH == ppc64le ]; then > + sudo apt-get install -y flex Why 'flex' is needed? Best regards, Ilya Maximets.
On 2019-10-29 09:52, Ilya Maximets wrote: > On 28.10.2019 22:22, David Wilder wrote: >> Add support for travis-ci ppc64le builds. >> >> - Updated matrix in .travis.yml to include a ppc64le build. >> - Added support to install packages needed by specific architectures. >> >> To keep the total build time at an acceptable level only a single >> build >> job is included in the matrix for ppc64le. >> >> A build report example can be found here [1] >> [0] >> https://urldefense.proofpoint.com/v2/url?u=http-3A__travis-2Dci.org_&d=DwICaQ&c=jf_iaSHvJObTbx-siA1ZOg&r=7ndxyKjH9UrBD68Us2WP1wI4BwEBQbeAyz8i_vwCCaI&m=6JANehIfGoxUMtwHhe4yob4UPeby0Y8ovgzRDIyJZFo&s=UMYL8rzJNph87seC0oJLBiWoe-sUSL80AJy0RMTgBzQ&e= >> [1] >> https://urldefense.proofpoint.com/v2/url?u=https-3A__travis-2Dci.org_djlwilder_ovs_builds_604098141&d=DwICaQ&c=jf_iaSHvJObTbx-siA1ZOg&r=7ndxyKjH9UrBD68Us2WP1wI4BwEBQbeAyz8i_vwCCaI&m=6JANehIfGoxUMtwHhe4yob4UPeby0Y8ovgzRDIyJZFo&s=pyd2yQpQ0snpwGE5El4RYZsatwl74sthM1KLqtIKCnY&e= >> Signed-off-by: David Wilder <dwilder@us.ibm.com> > > Hi David, > Thanks for working on this. I have a couple of question regarding > ppc64le support by TravisCI. It seems that they are not supporting > this architecture officially and refusing[1] to solve any issues that > appears while using it. There also no official documentation. > It's kind of a hidden feature that some projects are using for their > own risk [2]. Do you know why this happens or maybe you have some > insights about what is going on/how it works? Work is going on to increase ppc64le support on Travis by the end of the year. I dont have any details yet. My plan is to keep this to build-only ci until then. Important, ppc64le VM are only available on travis-ci.org, they are not available on travis-ci.com. > The API is also a bit strange because Travis started to officially > support arm builds, but this is done via 'arch' knob, not the 'os'. > Will it be changed over time for ppc64le? > Sorry, I dont know. > [1] > https://urldefense.proofpoint.com/v2/url?u=https-3A__travis-2Dci.community_t_ppc64le-2Darch-2Dsupport-2Don-2Dtravis-2Dci-2Dcom-2Dvs-2Dtravis-2Dci-2Dorg_2898_2&d=DwICaQ&c=jf_iaSHvJObTbx-siA1ZOg&r=7ndxyKjH9UrBD68Us2WP1wI4BwEBQbeAyz8i_vwCCaI&m=6JANehIfGoxUMtwHhe4yob4UPeby0Y8ovgzRDIyJZFo&s=TrXdSxjvnbbVQz7EzR5r0aE93lZMSdCiIUQT2wt8E3I&e= > [2] > https://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_openssl_openssl_commit_13da3ad00c80e1da816ca27f6c15b0ecee1bb0b8&d=DwICaQ&c=jf_iaSHvJObTbx-siA1ZOg&r=7ndxyKjH9UrBD68Us2WP1wI4BwEBQbeAyz8i_vwCCaI&m=6JANehIfGoxUMtwHhe4yob4UPeby0Y8ovgzRDIyJZFo&s=RWVuli-BT8E2IsW3rAA9MtqCVPZahNk8k7yqxEbgTT4&e= > > Few code comments inline. > >> --- >> .travis.yml | 5 +++-- >> .travis/linux-prepare.sh | 18 ++++++++++++++---- >> 2 files changed, 17 insertions(+), 6 deletions(-) >> >> diff --git a/.travis.yml b/.travis.yml >> index 5676d9748..c99f26815 100644 >> --- a/.travis.yml >> +++ b/.travis.yml >> @@ -14,7 +14,6 @@ addons: >> apt: >> packages: >> - bc >> - - gcc-multilib >> - libssl-dev >> - llvm-dev >> - libjemalloc1 >> @@ -24,7 +23,6 @@ addons: >> - libelf-dev >> - selinux-policy-dev >> - libunbound-dev >> - - libunbound-dev:i386 >> - libunwind-dev >> before_install: ./.travis/${TRAVIS_OS_NAME}-prepare.sh >> @@ -50,6 +48,9 @@ matrix: >> - os: osx >> compiler: clang >> env: OPTS="--disable-ssl" >> + - os: linux-ppc64le >> + compiler: gcc >> + env: OPTS="--disable-ssl" >> script: ./.travis/${TRAVIS_OS_NAME}-build.sh $OPTS >> diff --git a/.travis/linux-prepare.sh b/.travis/linux-prepare.sh >> index e546d32cb..f3a9a6d44 100755 >> --- a/.travis/linux-prepare.sh >> +++ b/.travis/linux-prepare.sh >> @@ -15,8 +15,18 @@ cd .. >> pip install --disable-pip-version-check --user six flake8 hacking >> pip install --user --upgrade docutils >> -if [ "$M32" ]; then >> - # 32-bit and 64-bit libunwind can not be installed at the same >> time. >> - # This will remove the 64-bit libunwind and install 32-bit >> version. >> - sudo apt-get install -y libunwind-dev:i386 >> +# Include packages needed by specific architectures. >> +if [ $TRAVIS_ARCH == amd64 ]; then >> + sudo apt-get install -y \ >> + libunbound-dev:i386 \ >> + gcc-multilib > > These packages are only needed for 32bit build, so you may just move > them > to the command that installs 32bit version of libunwind. > And since you're not building 32bit with ppc64le, the code could look > like: > > if [ "$M32" ]; then > # install 32 bit libs > fi > > if [ $TRAVIS_ARCH == ppc64le ]; then > # install ppc64le specific things. > fi Agreed. > >> + >> + if [ "$M32" ]; then >> + # 32-bit and 64-bit libunwind can not be installed at the same >> time. >> + # This will remove the 64-bit libunwind and install 32-bit >> version. >> + sudo apt-get install -y libunwind-dev:i386 >> + fi >> + >> +elif [ $TRAVIS_ARCH == ppc64le ]; then >> + sudo apt-get install -y flex > > Why 'flex' is needed? I found that Flex was needed for kernel builds, but since I am not doing that yet I can remove it. Thanks for the review, I will post a V2 patch shortly. > > Best regards, Ilya Maximets.
Hi, We are working to support arm64 build for ovs travis CI. It is indeed to use arch: arm64 to choose cpu architecture, because travis has provided native arm64 option now. But in this patch it seems ppc64 builds run on the ppc-VM + x86 native machine. Currently arm only select a part of jobs to run, which is defined in matrix:include. But the final object is to run all jobs. It means that arch: arm64 will be moved out of marix. If ppc plans to do the same in the future, it will conflict with arm jobs. Best Regards, Wei Yanqin -----Original Message----- From: ovs-dev-bounces@openvswitch.org <ovs-dev-bounces@openvswitch.org> On Behalf Of dwilder Sent: Wednesday, October 30, 2019 1:55 AM To: Ilya Maximets <i.maximets@ovn.org> Cc: ovs-dev@openvswitch.org; wilder@us.ibm.com Subject: Re: [ovs-dev] [PATCH ] travis: support ppc64le builds On 2019-10-29 09:52, Ilya Maximets wrote: > On 28.10.2019 22:22, David Wilder wrote: >> Add support for travis-ci ppc64le builds. >> >> - Updated matrix in .travis.yml to include a ppc64le build. >> - Added support to install packages needed by specific architectures. >> >> To keep the total build time at an acceptable level only a single >> build job is included in the matrix for ppc64le. >> >> A build report example can be found here [1] [0] >> https://urldefense.proofpoint.com/v2/url?u=http-3A__travis-2Dci.org_& >> d=DwICaQ&c=jf_iaSHvJObTbx-siA1ZOg&r=7ndxyKjH9UrBD68Us2WP1wI4BwEBQbeAy >> z8i_vwCCaI&m=6JANehIfGoxUMtwHhe4yob4UPeby0Y8ovgzRDIyJZFo&s=UMYL8rzJNp >> h87seC0oJLBiWoe-sUSL80AJy0RMTgBzQ&e= >> [1] >> https://urldefense.proofpoint.com/v2/url?u=https-3A__travis-2Dci.org_ >> djlwilder_ovs_builds_604098141&d=DwICaQ&c=jf_iaSHvJObTbx-siA1ZOg&r=7n >> dxyKjH9UrBD68Us2WP1wI4BwEBQbeAyz8i_vwCCaI&m=6JANehIfGoxUMtwHhe4yob4UP >> eby0Y8ovgzRDIyJZFo&s=pyd2yQpQ0snpwGE5El4RYZsatwl74sthM1KLqtIKCnY&e= >> Signed-off-by: David Wilder <dwilder@us.ibm.com> > > Hi David, > Thanks for working on this. I have a couple of question regarding > ppc64le support by TravisCI. It seems that they are not supporting > this architecture officially and refusing[1] to solve any issues that > appears while using it. There also no official documentation. > It's kind of a hidden feature that some projects are using for their > own risk [2]. Do you know why this happens or maybe you have some > insights about what is going on/how it works? Work is going on to increase ppc64le support on Travis by the end of the year. I dont have any details yet. My plan is to keep this to build-only ci until then. Important, ppc64le VM are only available on travis-ci.org, they are not available on travis-ci.com. > The API is also a bit strange because Travis started to officially > support arm builds, but this is done via 'arch' knob, not the 'os'. > Will it be changed over time for ppc64le? > Sorry, I dont know. > [1] > https://urldefense.proofpoint.com/v2/url?u=https-3A__travis-2Dci.commu > nity_t_ppc64le-2Darch-2Dsupport-2Don-2Dtravis-2Dci-2Dcom-2Dvs-2Dtravis > -2Dci-2Dorg_2898_2&d=DwICaQ&c=jf_iaSHvJObTbx-siA1ZOg&r=7ndxyKjH9UrBD68 > Us2WP1wI4BwEBQbeAyz8i_vwCCaI&m=6JANehIfGoxUMtwHhe4yob4UPeby0Y8ovgzRDIy > JZFo&s=TrXdSxjvnbbVQz7EzR5r0aE93lZMSdCiIUQT2wt8E3I&e= > [2] > https://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_openss > l_openssl_commit_13da3ad00c80e1da816ca27f6c15b0ecee1bb0b8&d=DwICaQ&c=j > f_iaSHvJObTbx-siA1ZOg&r=7ndxyKjH9UrBD68Us2WP1wI4BwEBQbeAyz8i_vwCCaI&m= > 6JANehIfGoxUMtwHhe4yob4UPeby0Y8ovgzRDIyJZFo&s=RWVuli-BT8E2IsW3rAA9MtqC > VPZahNk8k7yqxEbgTT4&e= > > Few code comments inline. > >> --- >> .travis.yml | 5 +++-- >> .travis/linux-prepare.sh | 18 ++++++++++++++---- >> 2 files changed, 17 insertions(+), 6 deletions(-) >> >> diff --git a/.travis.yml b/.travis.yml index 5676d9748..c99f26815 >> 100644 >> --- a/.travis.yml >> +++ b/.travis.yml >> @@ -14,7 +14,6 @@ addons: >> apt: >> packages: >> - bc >> - - gcc-multilib >> - libssl-dev >> - llvm-dev >> - libjemalloc1 >> @@ -24,7 +23,6 @@ addons: >> - libelf-dev >> - selinux-policy-dev >> - libunbound-dev >> - - libunbound-dev:i386 >> - libunwind-dev >> before_install: ./.travis/${TRAVIS_OS_NAME}-prepare.sh >> @@ -50,6 +48,9 @@ matrix: >> - os: osx >> compiler: clang >> env: OPTS="--disable-ssl" >> + - os: linux-ppc64le >> + compiler: gcc >> + env: OPTS="--disable-ssl" >> script: ./.travis/${TRAVIS_OS_NAME}-build.sh $OPTS >> diff --git a/.travis/linux-prepare.sh b/.travis/linux-prepare.sh >> index e546d32cb..f3a9a6d44 100755 >> --- a/.travis/linux-prepare.sh >> +++ b/.travis/linux-prepare.sh >> @@ -15,8 +15,18 @@ cd .. >> pip install --disable-pip-version-check --user six flake8 hacking >> pip install --user --upgrade docutils >> -if [ "$M32" ]; then >> - # 32-bit and 64-bit libunwind can not be installed at the same >> time. >> - # This will remove the 64-bit libunwind and install 32-bit >> version. >> - sudo apt-get install -y libunwind-dev:i386 >> +# Include packages needed by specific architectures. >> +if [ $TRAVIS_ARCH == amd64 ]; then >> + sudo apt-get install -y \ >> + libunbound-dev:i386 \ >> + gcc-multilib > > These packages are only needed for 32bit build, so you may just move > them to the command that installs 32bit version of libunwind. > And since you're not building 32bit with ppc64le, the code could look > like: > > if [ "$M32" ]; then > # install 32 bit libs > fi > > if [ $TRAVIS_ARCH == ppc64le ]; then > # install ppc64le specific things. > fi Agreed. > >> + >> + if [ "$M32" ]; then >> + # 32-bit and 64-bit libunwind can not be installed at the >> + same >> time. >> + # This will remove the 64-bit libunwind and install 32-bit >> version. >> + sudo apt-get install -y libunwind-dev:i386 >> + fi >> + >> +elif [ $TRAVIS_ARCH == ppc64le ]; then >> + sudo apt-get install -y flex > > Why 'flex' is needed? I found that Flex was needed for kernel builds, but since I am not doing that yet I can remove it. Thanks for the review, I will post a V2 patch shortly. > > Best regards, Ilya Maximets.
On 2019-10-30 19:04, Yanqin Wei (Arm Technology China) wrote: > Hi, > > We are working to support arm64 build for ovs travis CI. It is indeed > to use arch: arm64 to choose cpu architecture, because travis has > provided native arm64 option now. > But in this patch it seems ppc64 builds run on the ppc-VM + x86 native > machine. > Currently arm only select a part of jobs to run, which is defined in > matrix:include. But the final object is to run all jobs. It means that > arch: arm64 will be moved out of marix. If ppc plans to do the same > in the future, it will conflict with arm jobs. > > Best Regards, > Wei Yanqin Hi, I have added a build only test for ppc64le following the model used for osx. I think this is a good start for getting multi-arch support into Ci. I agree that running all jobs on the matrix on every arch is good goal. I dont completely understand your issue, is your concern the use of os: vs arch: ? I am glad to work with you to find a solution. Can you share your arm64 changes? We can discuss off-list if you prefer. > > -----Original Message----- > From: ovs-dev-bounces@openvswitch.org > <ovs-dev-bounces@openvswitch.org> On Behalf Of dwilder > Sent: Wednesday, October 30, 2019 1:55 AM > To: Ilya Maximets <i.maximets@ovn.org> > Cc: ovs-dev@openvswitch.org; wilder@us.ibm.com > Subject: Re: [ovs-dev] [PATCH ] travis: support ppc64le builds > > On 2019-10-29 09:52, Ilya Maximets wrote: >> On 28.10.2019 22:22, David Wilder wrote: >>> Add support for travis-ci ppc64le builds. >>> >>> - Updated matrix in .travis.yml to include a ppc64le build. >>> - Added support to install packages needed by specific architectures. >>> >>> To keep the total build time at an acceptable level only a single >>> build job is included in the matrix for ppc64le. >>> >>> A build report example can be found here [1] [0] >>> https://urldefense.proofpoint.com/v2/url?u=http-3A__travis-2Dci.org_& >>> d=DwICaQ&c=jf_iaSHvJObTbx-siA1ZOg&r=7ndxyKjH9UrBD68Us2WP1wI4BwEBQbeAy >>> z8i_vwCCaI&m=6JANehIfGoxUMtwHhe4yob4UPeby0Y8ovgzRDIyJZFo&s=UMYL8rzJNp >>> h87seC0oJLBiWoe-sUSL80AJy0RMTgBzQ&e= >>> [1] >>> https://urldefense.proofpoint.com/v2/url?u=https-3A__travis-2Dci.org_ >>> djlwilder_ovs_builds_604098141&d=DwICaQ&c=jf_iaSHvJObTbx-siA1ZOg&r=7n >>> dxyKjH9UrBD68Us2WP1wI4BwEBQbeAyz8i_vwCCaI&m=6JANehIfGoxUMtwHhe4yob4UP >>> eby0Y8ovgzRDIyJZFo&s=pyd2yQpQ0snpwGE5El4RYZsatwl74sthM1KLqtIKCnY&e= >>> Signed-off-by: David Wilder <dwilder@us.ibm.com> >> >> Hi David, >> Thanks for working on this. I have a couple of question regarding >> ppc64le support by TravisCI. It seems that they are not supporting >> this architecture officially and refusing[1] to solve any issues that >> appears while using it. There also no official documentation. >> It's kind of a hidden feature that some projects are using for their >> own risk [2]. Do you know why this happens or maybe you have some >> insights about what is going on/how it works? > > Work is going on to increase ppc64le support on Travis by the end of > the year. I dont have any details yet. My plan is to keep this to > build-only ci until then. Important, ppc64le VM are only available on > travis-ci.org, they are not available on travis-ci.com. > >> The API is also a bit strange because Travis started to officially >> support arm builds, but this is done via 'arch' knob, not the 'os'. >> Will it be changed over time for ppc64le? >> > > Sorry, I dont know. > >> [1] >> https://urldefense.proofpoint.com/v2/url?u=https-3A__travis-2Dci.commu >> nity_t_ppc64le-2Darch-2Dsupport-2Don-2Dtravis-2Dci-2Dcom-2Dvs-2Dtravis >> -2Dci-2Dorg_2898_2&d=DwICaQ&c=jf_iaSHvJObTbx-siA1ZOg&r=7ndxyKjH9UrBD68 >> Us2WP1wI4BwEBQbeAyz8i_vwCCaI&m=6JANehIfGoxUMtwHhe4yob4UPeby0Y8ovgzRDIy >> JZFo&s=TrXdSxjvnbbVQz7EzR5r0aE93lZMSdCiIUQT2wt8E3I&e= >> [2] >> https://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_openss >> l_openssl_commit_13da3ad00c80e1da816ca27f6c15b0ecee1bb0b8&d=DwICaQ&c=j >> f_iaSHvJObTbx-siA1ZOg&r=7ndxyKjH9UrBD68Us2WP1wI4BwEBQbeAyz8i_vwCCaI&m= >> 6JANehIfGoxUMtwHhe4yob4UPeby0Y8ovgzRDIyJZFo&s=RWVuli-BT8E2IsW3rAA9MtqC >> VPZahNk8k7yqxEbgTT4&e= >> >> Few code comments inline. >> >>> --- >>> .travis.yml | 5 +++-- >>> .travis/linux-prepare.sh | 18 ++++++++++++++---- >>> 2 files changed, 17 insertions(+), 6 deletions(-) >>> >>> diff --git a/.travis.yml b/.travis.yml index 5676d9748..c99f26815 >>> 100644 >>> --- a/.travis.yml >>> +++ b/.travis.yml >>> @@ -14,7 +14,6 @@ addons: >>> apt: >>> packages: >>> - bc >>> - - gcc-multilib >>> - libssl-dev >>> - llvm-dev >>> - libjemalloc1 >>> @@ -24,7 +23,6 @@ addons: >>> - libelf-dev >>> - selinux-policy-dev >>> - libunbound-dev >>> - - libunbound-dev:i386 >>> - libunwind-dev >>> before_install: ./.travis/${TRAVIS_OS_NAME}-prepare.sh >>> @@ -50,6 +48,9 @@ matrix: >>> - os: osx >>> compiler: clang >>> env: OPTS="--disable-ssl" >>> + - os: linux-ppc64le >>> + compiler: gcc >>> + env: OPTS="--disable-ssl" >>> script: ./.travis/${TRAVIS_OS_NAME}-build.sh $OPTS >>> diff --git a/.travis/linux-prepare.sh b/.travis/linux-prepare.sh >>> index e546d32cb..f3a9a6d44 100755 >>> --- a/.travis/linux-prepare.sh >>> +++ b/.travis/linux-prepare.sh >>> @@ -15,8 +15,18 @@ cd .. >>> pip install --disable-pip-version-check --user six flake8 hacking >>> pip install --user --upgrade docutils >>> -if [ "$M32" ]; then >>> - # 32-bit and 64-bit libunwind can not be installed at the same >>> time. >>> - # This will remove the 64-bit libunwind and install 32-bit >>> version. >>> - sudo apt-get install -y libunwind-dev:i386 >>> +# Include packages needed by specific architectures. >>> +if [ $TRAVIS_ARCH == amd64 ]; then >>> + sudo apt-get install -y \ >>> + libunbound-dev:i386 \ >>> + gcc-multilib >> >> These packages are only needed for 32bit build, so you may just move >> them to the command that installs 32bit version of libunwind. >> And since you're not building 32bit with ppc64le, the code could look >> like: >> >> if [ "$M32" ]; then >> # install 32 bit libs >> fi >> >> if [ $TRAVIS_ARCH == ppc64le ]; then >> # install ppc64le specific things. >> fi > > Agreed. > >> >>> + >>> + if [ "$M32" ]; then >>> + # 32-bit and 64-bit libunwind can not be installed at the >>> + same >>> time. >>> + # This will remove the 64-bit libunwind and install 32-bit >>> version. >>> + sudo apt-get install -y libunwind-dev:i386 >>> + fi >>> + >>> +elif [ $TRAVIS_ARCH == ppc64le ]; then >>> + sudo apt-get install -y flex >> >> Why 'flex' is needed? > > I found that Flex was needed for kernel builds, but since I am not > doing that yet I can remove it. > > Thanks for the review, I will post a V2 patch shortly. > >> >> Best regards, Ilya Maximets. > _______________________________________________ > dev mailing list > dev@openvswitch.org > https://urldefense.proofpoint.com/v2/url?u=https-3A__mail.openvswitch.org_mailman_listinfo_ovs-2Ddev&d=DwIFAg&c=jf_iaSHvJObTbx-siA1ZOg&r=7ndxyKjH9UrBD68Us2WP1wI4BwEBQbeAyz8i_vwCCaI&m=_QMOOwopXhBHpUCeEDvwONjciJxP_WEksZXnZOBIFHM&s=W1XHbP-_Wjy0FfaSXS37Lx5F-tHW0oK9VRLwt5YpMro&e=
Hi David,
Thanks for your reply. Yes, my concern is how to define arch and os in .travis.yml after we cover all builds and cases for arm and ppc.
This pattern can enable all builds and testsuits for x86 and arm
arch:
- amd64
- arm64
os:
- linux
This can enable all jobs for x86 and ppc.
arch:
- amd64 //default
os:
- linux
- linux-ppc64le
But it does not work to combine them. This means four kinds of arch+os combinations in all. Arm64+linux-ppc64le is invalid.
arch:
- amd64
- arm64
os:
- linux
- linux-ppc64le
So if we finally cover all the builds and cases for arm/ppc, we have to duplicate all jobs for different cpu arch in the matrix include.
matrix:
include:
- os: linux-ppc64le
env: job1
- os: linux-ppc64le
env: job2
...
- arch: arm64
env: job1
- arch: arm64
env: job2
...
But currently either arm or ppc has not cover all the cases, so they can coexist in build-matrix. And there is no conflict in build/prepare scripts, because both of them use TRAVIS_ARCH variable to indicate cpu arch.
The patch series to enable arm CI is under internal review. It will be submitted when ready.
Best Regards,
Wei Yanqin
-----Original Message-----
From: dwilder <dwilder@us.ibm.com>
Sent: Saturday, November 2, 2019 1:09 AM
To: Yanqin Wei (Arm Technology China) <Yanqin.Wei@arm.com>
Cc: Ilya Maximets <i.maximets@ovn.org>; ovs-dev@openvswitch.org; wilder@us.ibm.com; nd <nd@arm.com>
Subject: Re: RE: [ovs-dev] [PATCH ] travis: support ppc64le builds
On 2019-10-30 19:04, Yanqin Wei (Arm Technology China) wrote:
> Hi,
>
> We are working to support arm64 build for ovs travis CI. It is indeed
> to use arch: arm64 to choose cpu architecture, because travis has
> provided native arm64 option now.
> But in this patch it seems ppc64 builds run on the ppc-VM + x86 native
> machine.
> Currently arm only select a part of jobs to run, which is defined in
> matrix:include. But the final object is to run all jobs. It means that
> arch: arm64 will be moved out of marix. If ppc plans to do the same
> in the future, it will conflict with arm jobs.
>
> Best Regards,
> Wei Yanqin
Hi,
I have added a build only test for ppc64le following the model used for osx. I think this is a good start for getting multi-arch support into Ci.
I agree that running all jobs on the matrix on every arch is good goal.
I dont completely understand your issue, is your concern the use of os:
vs arch: ?
I am glad to work with you to find a solution. Can you share your arm64 changes? We can discuss off-list if you prefer.
>
> -----Original Message-----
> From: ovs-dev-bounces@openvswitch.org
> <ovs-dev-bounces@openvswitch.org> On Behalf Of dwilder
> Sent: Wednesday, October 30, 2019 1:55 AM
> To: Ilya Maximets <i.maximets@ovn.org>
> Cc: ovs-dev@openvswitch.org; wilder@us.ibm.com
> Subject: Re: [ovs-dev] [PATCH ] travis: support ppc64le builds
>
> On 2019-10-29 09:52, Ilya Maximets wrote:
>> On 28.10.2019 22:22, David Wilder wrote:
>>> Add support for travis-ci ppc64le builds.
>>>
>>> - Updated matrix in .travis.yml to include a ppc64le build.
>>> - Added support to install packages needed by specific architectures.
>>>
>>> To keep the total build time at an acceptable level only a single
>>> build job is included in the matrix for ppc64le.
>>>
>>> A build report example can be found here [1] [0]
>>> https://urldefense.proofpoint.com/v2/url?u=http-3A__travis-2Dci.org_&
>>> d=DwICaQ&c=jf_iaSHvJObTbx-siA1ZOg&r=7ndxyKjH9UrBD68Us2WP1wI4BwEBQbeAy
>>> z8i_vwCCaI&m=6JANehIfGoxUMtwHhe4yob4UPeby0Y8ovgzRDIyJZFo&s=UMYL8rzJNp
>>> h87seC0oJLBiWoe-sUSL80AJy0RMTgBzQ&e=
>>> [1]
>>> https://urldefense.proofpoint.com/v2/url?u=https-3A__travis-2Dci.org_
>>> djlwilder_ovs_builds_604098141&d=DwICaQ&c=jf_iaSHvJObTbx-siA1ZOg&r=7n
>>> dxyKjH9UrBD68Us2WP1wI4BwEBQbeAyz8i_vwCCaI&m=6JANehIfGoxUMtwHhe4yob4UP
>>> eby0Y8ovgzRDIyJZFo&s=pyd2yQpQ0snpwGE5El4RYZsatwl74sthM1KLqtIKCnY&e=
>>> Signed-off-by: David Wilder <dwilder@us.ibm.com>
>>
>> Hi David,
>> Thanks for working on this. I have a couple of question regarding
>> ppc64le support by TravisCI. It seems that they are not supporting
>> this architecture officially and refusing[1] to solve any issues that
>> appears while using it. There also no official documentation.
>> It's kind of a hidden feature that some projects are using for their
>> own risk [2]. Do you know why this happens or maybe you have some
>> insights about what is going on/how it works?
>
> Work is going on to increase ppc64le support on Travis by the end of
> the year. I dont have any details yet. My plan is to keep this to
> build-only ci until then. Important, ppc64le VM are only available on
> travis-ci.org, they are not available on travis-ci.com.
>
>> The API is also a bit strange because Travis started to officially
>> support arm builds, but this is done via 'arch' knob, not the 'os'.
>> Will it be changed over time for ppc64le?
>>
>
> Sorry, I dont know.
>
>> [1]
>> https://urldefense.proofpoint.com/v2/url?u=https-3A__travis-2Dci.commu
>> nity_t_ppc64le-2Darch-2Dsupport-2Don-2Dtravis-2Dci-2Dcom-2Dvs-2Dtravis
>> -2Dci-2Dorg_2898_2&d=DwICaQ&c=jf_iaSHvJObTbx-siA1ZOg&r=7ndxyKjH9UrBD68
>> Us2WP1wI4BwEBQbeAyz8i_vwCCaI&m=6JANehIfGoxUMtwHhe4yob4UPeby0Y8ovgzRDIy
>> JZFo&s=TrXdSxjvnbbVQz7EzR5r0aE93lZMSdCiIUQT2wt8E3I&e=
>> [2]
>> https://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_openss
>> l_openssl_commit_13da3ad00c80e1da816ca27f6c15b0ecee1bb0b8&d=DwICaQ&c=j
>> f_iaSHvJObTbx-siA1ZOg&r=7ndxyKjH9UrBD68Us2WP1wI4BwEBQbeAyz8i_vwCCaI&m=
>> 6JANehIfGoxUMtwHhe4yob4UPeby0Y8ovgzRDIyJZFo&s=RWVuli-BT8E2IsW3rAA9MtqC
>> VPZahNk8k7yqxEbgTT4&e=
>>
>> Few code comments inline.
>>
>>> ---
>>> .travis.yml | 5 +++--
>>> .travis/linux-prepare.sh | 18 ++++++++++++++----
>>> 2 files changed, 17 insertions(+), 6 deletions(-)
>>>
>>> diff --git a/.travis.yml b/.travis.yml index 5676d9748..c99f26815
>>> 100644
>>> --- a/.travis.yml
>>> +++ b/.travis.yml
>>> @@ -14,7 +14,6 @@ addons:
>>> apt:
>>> packages:
>>> - bc
>>> - - gcc-multilib
>>> - libssl-dev
>>> - llvm-dev
>>> - libjemalloc1
>>> @@ -24,7 +23,6 @@ addons:
>>> - libelf-dev
>>> - selinux-policy-dev
>>> - libunbound-dev
>>> - - libunbound-dev:i386
>>> - libunwind-dev
>>> before_install: ./.travis/${TRAVIS_OS_NAME}-prepare.sh
>>> @@ -50,6 +48,9 @@ matrix:
>>> - os: osx
>>> compiler: clang
>>> env: OPTS="--disable-ssl"
>>> + - os: linux-ppc64le
>>> + compiler: gcc
>>> + env: OPTS="--disable-ssl"
>>> script: ./.travis/${TRAVIS_OS_NAME}-build.sh $OPTS
>>> diff --git a/.travis/linux-prepare.sh b/.travis/linux-prepare.sh
>>> index e546d32cb..f3a9a6d44 100755
>>> --- a/.travis/linux-prepare.sh
>>> +++ b/.travis/linux-prepare.sh
>>> @@ -15,8 +15,18 @@ cd ..
>>> pip install --disable-pip-version-check --user six flake8 hacking
>>> pip install --user --upgrade docutils
>>> -if [ "$M32" ]; then
>>> - # 32-bit and 64-bit libunwind can not be installed at the same
>>> time.
>>> - # This will remove the 64-bit libunwind and install 32-bit
>>> version.
>>> - sudo apt-get install -y libunwind-dev:i386
>>> +# Include packages needed by specific architectures.
>>> +if [ $TRAVIS_ARCH == amd64 ]; then
>>> + sudo apt-get install -y \
>>> + libunbound-dev:i386 \
>>> + gcc-multilib
>>
>> These packages are only needed for 32bit build, so you may just move
>> them to the command that installs 32bit version of libunwind.
>> And since you're not building 32bit with ppc64le, the code could look
>> like:
>>
>> if [ "$M32" ]; then
>> # install 32 bit libs
>> fi
>>
>> if [ $TRAVIS_ARCH == ppc64le ]; then
>> # install ppc64le specific things.
>> fi
>
> Agreed.
>
>>
>>> +
>>> + if [ "$M32" ]; then
>>> + # 32-bit and 64-bit libunwind can not be installed at the
>>> + same
>>> time.
>>> + # This will remove the 64-bit libunwind and install 32-bit
>>> version.
>>> + sudo apt-get install -y libunwind-dev:i386
>>> + fi
>>> +
>>> +elif [ $TRAVIS_ARCH == ppc64le ]; then
>>> + sudo apt-get install -y flex
>>
>> Why 'flex' is needed?
>
> I found that Flex was needed for kernel builds, but since I am not
> doing that yet I can remove it.
>
> Thanks for the review, I will post a V2 patch shortly.
>
>>
>> Best regards, Ilya Maximets.
> _______________________________________________
> dev mailing list
> dev@openvswitch.org
> https://urldefense.proofpoint.com/v2/url?u=https-3A__mail.openvswitch.org_mailman_listinfo_ovs-2Ddev&d=DwIFAg&c=jf_iaSHvJObTbx-siA1ZOg&r=7ndxyKjH9UrBD68Us2WP1wI4BwEBQbeAyz8i_vwCCaI&m=_QMOOwopXhBHpUCeEDvwONjciJxP_WEksZXnZOBIFHM&s=W1XHbP-_Wjy0FfaSXS37Lx5F-tHW0oK9VRLwt5YpMro&e=
Hi Wei
If I change my matrix:include to use "arch: ppc64le" rather than "os:
linux-ppc64le", will it eliminate your concern?
matrix:
include:
- - os: linux-ppc64le
+ - arch: ppc64le
compiler: gcc
env: OPTS="--disable-ssl"
Later when we want to enable the full matrix on all arch we would add:
1) arch:
- amd64
- ppc64le
- arm64
2) eliminate the include: -arch: ppc64le
3) Add any exclude: - arch:XXX sections for any tests that dont apply.
For example I would add:
exclude:
- arch: ppc64le
env: M32=1 OPTS="--disable-ssl"
Regards,
David
On 2019-11-02 02:09, Yanqin Wei (Arm Technology China) wrote:
> Hi David,
>
> Thanks for your reply. Yes, my concern is how to define arch and os
> in .travis.yml after we cover all builds and cases for arm and ppc.
> This pattern can enable all builds and testsuits for x86 and arm
> arch:
> - amd64
> - arm64
> os:
> - linux
>
> This can enable all jobs for x86 and ppc.
> arch:
> - amd64 //default
> os:
> - linux
> - linux-ppc64le
>
> But it does not work to combine them. This means four kinds of
> arch+os combinations in all. Arm64+linux-ppc64le is invalid.
> arch:
> - amd64
> - arm64
> os:
> - linux
> - linux-ppc64le
>
> So if we finally cover all the builds and cases for arm/ppc, we have
> to duplicate all jobs for different cpu arch in the matrix include.
> matrix:
> include:
> - os: linux-ppc64le
> env: job1
> - os: linux-ppc64le
> env: job2
> ...
> - arch: arm64
> env: job1
> - arch: arm64
> env: job2
> ...
>
> But currently either arm or ppc has not cover all the cases, so they
> can coexist in build-matrix. And there is no conflict in
> build/prepare scripts, because both of them use TRAVIS_ARCH variable
> to indicate cpu arch.
> The patch series to enable arm CI is under internal review. It will be
> submitted when ready.
>
> Best Regards,
> Wei Yanqin
>
>
> -----Original Message-----
> From: dwilder <dwilder@us.ibm.com>
> Sent: Saturday, November 2, 2019 1:09 AM
> To: Yanqin Wei (Arm Technology China) <Yanqin.Wei@arm.com>
> Cc: Ilya Maximets <i.maximets@ovn.org>; ovs-dev@openvswitch.org;
> wilder@us.ibm.com; nd <nd@arm.com>
> Subject: Re: RE: [ovs-dev] [PATCH ] travis: support ppc64le builds
>
> On 2019-10-30 19:04, Yanqin Wei (Arm Technology China) wrote:
>> Hi,
>>
>> We are working to support arm64 build for ovs travis CI. It is indeed
>> to use arch: arm64 to choose cpu architecture, because travis has
>> provided native arm64 option now.
>> But in this patch it seems ppc64 builds run on the ppc-VM + x86 native
>> machine.
>> Currently arm only select a part of jobs to run, which is defined in
>> matrix:include. But the final object is to run all jobs. It means that
>> arch: arm64 will be moved out of marix. If ppc plans to do the same
>> in the future, it will conflict with arm jobs.
>>
>> Best Regards,
>> Wei Yanqin
>
> Hi,
> I have added a build only test for ppc64le following the model used
> for osx. I think this is a good start for getting multi-arch support
> into Ci.
>
> I agree that running all jobs on the matrix on every arch is good goal.
> I dont completely understand your issue, is your concern the use of os:
> vs arch: ?
>
> I am glad to work with you to find a solution. Can you share your
> arm64 changes? We can discuss off-list if you prefer.
>
>
>>
>> -----Original Message-----
>> From: ovs-dev-bounces@openvswitch.org
>> <ovs-dev-bounces@openvswitch.org> On Behalf Of dwilder
>> Sent: Wednesday, October 30, 2019 1:55 AM
>> To: Ilya Maximets <i.maximets@ovn.org>
>> Cc: ovs-dev@openvswitch.org; wilder@us.ibm.com
>> Subject: Re: [ovs-dev] [PATCH ] travis: support ppc64le builds
>>
>> On 2019-10-29 09:52, Ilya Maximets wrote:
>>> On 28.10.2019 22:22, David Wilder wrote:
>>>> Add support for travis-ci ppc64le builds.
>>>>
>>>> - Updated matrix in .travis.yml to include a ppc64le build.
>>>> - Added support to install packages needed by specific
>>>> architectures.
>>>>
>>>> To keep the total build time at an acceptable level only a single
>>>> build job is included in the matrix for ppc64le.
>>>>
>>>> A build report example can be found here [1] [0]
>>>> https://urldefense.proofpoint.com/v2/url?u=http-3A__travis-2Dci.org_&
>>>> d=DwICaQ&c=jf_iaSHvJObTbx-siA1ZOg&r=7ndxyKjH9UrBD68Us2WP1wI4BwEBQbeAy
>>>> z8i_vwCCaI&m=6JANehIfGoxUMtwHhe4yob4UPeby0Y8ovgzRDIyJZFo&s=UMYL8rzJNp
>>>> h87seC0oJLBiWoe-sUSL80AJy0RMTgBzQ&e=
>>>> [1]
>>>> https://urldefense.proofpoint.com/v2/url?u=https-3A__travis-2Dci.org_
>>>> djlwilder_ovs_builds_604098141&d=DwICaQ&c=jf_iaSHvJObTbx-siA1ZOg&r=7n
>>>> dxyKjH9UrBD68Us2WP1wI4BwEBQbeAyz8i_vwCCaI&m=6JANehIfGoxUMtwHhe4yob4UP
>>>> eby0Y8ovgzRDIyJZFo&s=pyd2yQpQ0snpwGE5El4RYZsatwl74sthM1KLqtIKCnY&e=
>>>> Signed-off-by: David Wilder <dwilder@us.ibm.com>
>>>
>>> Hi David,
>>> Thanks for working on this. I have a couple of question regarding
>>> ppc64le support by TravisCI. It seems that they are not supporting
>>> this architecture officially and refusing[1] to solve any issues that
>>> appears while using it. There also no official documentation.
>>> It's kind of a hidden feature that some projects are using for their
>>> own risk [2]. Do you know why this happens or maybe you have some
>>> insights about what is going on/how it works?
>>
>> Work is going on to increase ppc64le support on Travis by the end of
>> the year. I dont have any details yet. My plan is to keep this to
>> build-only ci until then. Important, ppc64le VM are only available on
>> travis-ci.org, they are not available on travis-ci.com.
>>
>>> The API is also a bit strange because Travis started to officially
>>> support arm builds, but this is done via 'arch' knob, not the 'os'.
>>> Will it be changed over time for ppc64le?
>>>
>>
>> Sorry, I dont know.
>>
>>> [1]
>>> https://urldefense.proofpoint.com/v2/url?u=https-3A__travis-2Dci.commu
>>> nity_t_ppc64le-2Darch-2Dsupport-2Don-2Dtravis-2Dci-2Dcom-2Dvs-2Dtravis
>>> -2Dci-2Dorg_2898_2&d=DwICaQ&c=jf_iaSHvJObTbx-siA1ZOg&r=7ndxyKjH9UrBD68
>>> Us2WP1wI4BwEBQbeAyz8i_vwCCaI&m=6JANehIfGoxUMtwHhe4yob4UPeby0Y8ovgzRDIy
>>> JZFo&s=TrXdSxjvnbbVQz7EzR5r0aE93lZMSdCiIUQT2wt8E3I&e=
>>> [2]
>>> https://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_openss
>>> l_openssl_commit_13da3ad00c80e1da816ca27f6c15b0ecee1bb0b8&d=DwICaQ&c=j
>>> f_iaSHvJObTbx-siA1ZOg&r=7ndxyKjH9UrBD68Us2WP1wI4BwEBQbeAyz8i_vwCCaI&m=
>>> 6JANehIfGoxUMtwHhe4yob4UPeby0Y8ovgzRDIyJZFo&s=RWVuli-BT8E2IsW3rAA9MtqC
>>> VPZahNk8k7yqxEbgTT4&e=
>>>
>>> Few code comments inline.
>>>
>>>> ---
>>>> .travis.yml | 5 +++--
>>>> .travis/linux-prepare.sh | 18 ++++++++++++++----
>>>> 2 files changed, 17 insertions(+), 6 deletions(-)
>>>>
>>>> diff --git a/.travis.yml b/.travis.yml index 5676d9748..c99f26815
>>>> 100644
>>>> --- a/.travis.yml
>>>> +++ b/.travis.yml
>>>> @@ -14,7 +14,6 @@ addons:
>>>> apt:
>>>> packages:
>>>> - bc
>>>> - - gcc-multilib
>>>> - libssl-dev
>>>> - llvm-dev
>>>> - libjemalloc1
>>>> @@ -24,7 +23,6 @@ addons:
>>>> - libelf-dev
>>>> - selinux-policy-dev
>>>> - libunbound-dev
>>>> - - libunbound-dev:i386
>>>> - libunwind-dev
>>>> before_install: ./.travis/${TRAVIS_OS_NAME}-prepare.sh
>>>> @@ -50,6 +48,9 @@ matrix:
>>>> - os: osx
>>>> compiler: clang
>>>> env: OPTS="--disable-ssl"
>>>> + - os: linux-ppc64le
>>>> + compiler: gcc
>>>> + env: OPTS="--disable-ssl"
>>>> script: ./.travis/${TRAVIS_OS_NAME}-build.sh $OPTS
>>>> diff --git a/.travis/linux-prepare.sh b/.travis/linux-prepare.sh
>>>> index e546d32cb..f3a9a6d44 100755
>>>> --- a/.travis/linux-prepare.sh
>>>> +++ b/.travis/linux-prepare.sh
>>>> @@ -15,8 +15,18 @@ cd ..
>>>> pip install --disable-pip-version-check --user six flake8 hacking
>>>> pip install --user --upgrade docutils
>>>> -if [ "$M32" ]; then
>>>> - # 32-bit and 64-bit libunwind can not be installed at the same
>>>> time.
>>>> - # This will remove the 64-bit libunwind and install 32-bit
>>>> version.
>>>> - sudo apt-get install -y libunwind-dev:i386
>>>> +# Include packages needed by specific architectures.
>>>> +if [ $TRAVIS_ARCH == amd64 ]; then
>>>> + sudo apt-get install -y \
>>>> + libunbound-dev:i386 \
>>>> + gcc-multilib
>>>
>>> These packages are only needed for 32bit build, so you may just move
>>> them to the command that installs 32bit version of libunwind.
>>> And since you're not building 32bit with ppc64le, the code could look
>>> like:
>>>
>>> if [ "$M32" ]; then
>>> # install 32 bit libs
>>> fi
>>>
>>> if [ $TRAVIS_ARCH == ppc64le ]; then
>>> # install ppc64le specific things.
>>> fi
>>
>> Agreed.
>>
>>>
>>>> +
>>>> + if [ "$M32" ]; then
>>>> + # 32-bit and 64-bit libunwind can not be installed at the
>>>> + same
>>>> time.
>>>> + # This will remove the 64-bit libunwind and install 32-bit
>>>> version.
>>>> + sudo apt-get install -y libunwind-dev:i386
>>>> + fi
>>>> +
>>>> +elif [ $TRAVIS_ARCH == ppc64le ]; then
>>>> + sudo apt-get install -y flex
>>>
>>> Why 'flex' is needed?
>>
>> I found that Flex was needed for kernel builds, but since I am not
>> doing that yet I can remove it.
>>
>> Thanks for the review, I will post a V2 patch shortly.
>>
>>>
>>> Best regards, Ilya Maximets.
>> _______________________________________________
>> dev mailing list
>> dev@openvswitch.org
>> https://urldefense.proofpoint.com/v2/url?u=https-3A__mail.openvswitch.org_mailman_listinfo_ovs-2Ddev&d=DwIFAg&c=jf_iaSHvJObTbx-siA1ZOg&r=7ndxyKjH9UrBD68Us2WP1wI4BwEBQbeAyz8i_vwCCaI&m=_QMOOwopXhBHpUCeEDvwONjciJxP_WEksZXnZOBIFHM&s=W1XHbP-_Wjy0FfaSXS37Lx5F-tHW0oK9VRLwt5YpMro&e=
Hi David, I am sorry to reply this late. Yes, if travis support this configuration, it should be a good solution for me. Best regards, Wei Yanqin -----Original Message----- From: dwilder <dwilder@us.ibm.com> Sent: Wednesday, November 6, 2019 4:32 AM To: Yanqin Wei (Arm Technology China) <Yanqin.Wei@arm.com> Cc: Ilya Maximets <i.maximets@ovn.org>; ovs-dev@openvswitch.org; wilder@us.ibm.com; nd <nd@arm.com> Subject: Re: RE: RE: [ovs-dev] [PATCH ] travis: support ppc64le builds Hi Wei If I change my matrix:include to use "arch: ppc64le" rather than "os: linux-ppc64le", will it eliminate your concern? matrix: include: - - os: linux-ppc64le + - arch: ppc64le compiler: gcc env: OPTS="--disable-ssl" Later when we want to enable the full matrix on all arch we would add: 1) arch: - amd64 - ppc64le - arm64 2) eliminate the include: -arch: ppc64le 3) Add any exclude: - arch:XXX sections for any tests that dont apply. For example I would add: exclude: - arch: ppc64le env: M32=1 OPTS="--disable-ssl" Regards, David On 2019-11-02 02:09, Yanqin Wei (Arm Technology China) wrote: > Hi David, > > Thanks for your reply. Yes, my concern is how to define arch and os > in .travis.yml after we cover all builds and cases for arm and ppc. > This pattern can enable all builds and testsuits for x86 and arm > arch: > - amd64 > - arm64 > os: > - linux > > This can enable all jobs for x86 and ppc. > arch: > - amd64 //default > os: > - linux > - linux-ppc64le > > But it does not work to combine them. This means four kinds of > arch+os combinations in all. Arm64+linux-ppc64le is invalid. > arch: > - amd64 > - arm64 > os: > - linux > - linux-ppc64le > > So if we finally cover all the builds and cases for arm/ppc, we have > to duplicate all jobs for different cpu arch in the matrix include. > matrix: > include: > - os: linux-ppc64le > env: job1 > - os: linux-ppc64le > env: job2 > ... > - arch: arm64 > env: job1 > - arch: arm64 > env: job2 > ... > > But currently either arm or ppc has not cover all the cases, so they > can coexist in build-matrix. And there is no conflict in > build/prepare scripts, because both of them use TRAVIS_ARCH variable > to indicate cpu arch. > The patch series to enable arm CI is under internal review. It will be > submitted when ready. > > Best Regards, > Wei Yanqin > > > -----Original Message----- > From: dwilder <dwilder@us.ibm.com> > Sent: Saturday, November 2, 2019 1:09 AM > To: Yanqin Wei (Arm Technology China) <Yanqin.Wei@arm.com> > Cc: Ilya Maximets <i.maximets@ovn.org>; ovs-dev@openvswitch.org; > wilder@us.ibm.com; nd <nd@arm.com> > Subject: Re: RE: [ovs-dev] [PATCH ] travis: support ppc64le builds > > On 2019-10-30 19:04, Yanqin Wei (Arm Technology China) wrote: >> Hi, >> >> We are working to support arm64 build for ovs travis CI. It is indeed >> to use arch: arm64 to choose cpu architecture, because travis has >> provided native arm64 option now. >> But in this patch it seems ppc64 builds run on the ppc-VM + x86 >> native machine. >> Currently arm only select a part of jobs to run, which is defined in >> matrix:include. But the final object is to run all jobs. It means >> that >> arch: arm64 will be moved out of marix. If ppc plans to do the same >> in the future, it will conflict with arm jobs. >> >> Best Regards, >> Wei Yanqin > > Hi, > I have added a build only test for ppc64le following the model used > for osx. I think this is a good start for getting multi-arch support > into Ci. > > I agree that running all jobs on the matrix on every arch is good goal. > I dont completely understand your issue, is your concern the use of os: > vs arch: ? > > I am glad to work with you to find a solution. Can you share your > arm64 changes? We can discuss off-list if you prefer. > > >> >> -----Original Message----- >> From: ovs-dev-bounces@openvswitch.org >> <ovs-dev-bounces@openvswitch.org> On Behalf Of dwilder >> Sent: Wednesday, October 30, 2019 1:55 AM >> To: Ilya Maximets <i.maximets@ovn.org> >> Cc: ovs-dev@openvswitch.org; wilder@us.ibm.com >> Subject: Re: [ovs-dev] [PATCH ] travis: support ppc64le builds >> >> On 2019-10-29 09:52, Ilya Maximets wrote: >>> On 28.10.2019 22:22, David Wilder wrote: >>>> Add support for travis-ci ppc64le builds. >>>> >>>> - Updated matrix in .travis.yml to include a ppc64le build. >>>> - Added support to install packages needed by specific >>>> architectures. >>>> >>>> To keep the total build time at an acceptable level only a single >>>> build job is included in the matrix for ppc64le. >>>> >>>> A build report example can be found here [1] [0] >>>> https://urldefense.proofpoint.com/v2/url?u=http-3A__travis-2Dci.org >>>> _& >>>> d=DwICaQ&c=jf_iaSHvJObTbx-siA1ZOg&r=7ndxyKjH9UrBD68Us2WP1wI4BwEBQbe >>>> Ay >>>> z8i_vwCCaI&m=6JANehIfGoxUMtwHhe4yob4UPeby0Y8ovgzRDIyJZFo&s=UMYL8rzJ >>>> Np h87seC0oJLBiWoe-sUSL80AJy0RMTgBzQ&e= >>>> [1] >>>> https://urldefense.proofpoint.com/v2/url?u=https-3A__travis-2Dci.or >>>> g_ >>>> djlwilder_ovs_builds_604098141&d=DwICaQ&c=jf_iaSHvJObTbx-siA1ZOg&r= >>>> 7n >>>> dxyKjH9UrBD68Us2WP1wI4BwEBQbeAyz8i_vwCCaI&m=6JANehIfGoxUMtwHhe4yob4 >>>> UP >>>> eby0Y8ovgzRDIyJZFo&s=pyd2yQpQ0snpwGE5El4RYZsatwl74sthM1KLqtIKCnY&e= >>>> Signed-off-by: David Wilder <dwilder@us.ibm.com> >>> >>> Hi David, >>> Thanks for working on this. I have a couple of question regarding >>> ppc64le support by TravisCI. It seems that they are not supporting >>> this architecture officially and refusing[1] to solve any issues >>> that appears while using it. There also no official documentation. >>> It's kind of a hidden feature that some projects are using for their >>> own risk [2]. Do you know why this happens or maybe you have some >>> insights about what is going on/how it works? >> >> Work is going on to increase ppc64le support on Travis by the end of >> the year. I dont have any details yet. My plan is to keep this to >> build-only ci until then. Important, ppc64le VM are only available >> on travis-ci.org, they are not available on travis-ci.com. >> >>> The API is also a bit strange because Travis started to officially >>> support arm builds, but this is done via 'arch' knob, not the 'os'. >>> Will it be changed over time for ppc64le? >>> >> >> Sorry, I dont know. >> >>> [1] >>> https://urldefense.proofpoint.com/v2/url?u=https-3A__travis-2Dci.com >>> mu >>> nity_t_ppc64le-2Darch-2Dsupport-2Don-2Dtravis-2Dci-2Dcom-2Dvs-2Dtrav >>> is >>> -2Dci-2Dorg_2898_2&d=DwICaQ&c=jf_iaSHvJObTbx-siA1ZOg&r=7ndxyKjH9UrBD >>> 68 >>> Us2WP1wI4BwEBQbeAyz8i_vwCCaI&m=6JANehIfGoxUMtwHhe4yob4UPeby0Y8ovgzRD >>> Iy JZFo&s=TrXdSxjvnbbVQz7EzR5r0aE93lZMSdCiIUQT2wt8E3I&e= >>> [2] >>> https://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_open >>> ss >>> l_openssl_commit_13da3ad00c80e1da816ca27f6c15b0ecee1bb0b8&d=DwICaQ&c >>> =j >>> f_iaSHvJObTbx-siA1ZOg&r=7ndxyKjH9UrBD68Us2WP1wI4BwEBQbeAyz8i_vwCCaI& >>> m= >>> 6JANehIfGoxUMtwHhe4yob4UPeby0Y8ovgzRDIyJZFo&s=RWVuli-BT8E2IsW3rAA9Mt >>> qC >>> VPZahNk8k7yqxEbgTT4&e= >>> >>> Few code comments inline. >>> >>>> --- >>>> .travis.yml | 5 +++-- >>>> .travis/linux-prepare.sh | 18 ++++++++++++++---- >>>> 2 files changed, 17 insertions(+), 6 deletions(-) >>>> >>>> diff --git a/.travis.yml b/.travis.yml index 5676d9748..c99f26815 >>>> 100644 >>>> --- a/.travis.yml >>>> +++ b/.travis.yml >>>> @@ -14,7 +14,6 @@ addons: >>>> apt: >>>> packages: >>>> - bc >>>> - - gcc-multilib >>>> - libssl-dev >>>> - llvm-dev >>>> - libjemalloc1 >>>> @@ -24,7 +23,6 @@ addons: >>>> - libelf-dev >>>> - selinux-policy-dev >>>> - libunbound-dev >>>> - - libunbound-dev:i386 >>>> - libunwind-dev >>>> before_install: ./.travis/${TRAVIS_OS_NAME}-prepare.sh >>>> @@ -50,6 +48,9 @@ matrix: >>>> - os: osx >>>> compiler: clang >>>> env: OPTS="--disable-ssl" >>>> + - os: linux-ppc64le >>>> + compiler: gcc >>>> + env: OPTS="--disable-ssl" >>>> script: ./.travis/${TRAVIS_OS_NAME}-build.sh $OPTS >>>> diff --git a/.travis/linux-prepare.sh b/.travis/linux-prepare.sh >>>> index e546d32cb..f3a9a6d44 100755 >>>> --- a/.travis/linux-prepare.sh >>>> +++ b/.travis/linux-prepare.sh >>>> @@ -15,8 +15,18 @@ cd .. >>>> pip install --disable-pip-version-check --user six flake8 hacking >>>> pip install --user --upgrade docutils >>>> -if [ "$M32" ]; then >>>> - # 32-bit and 64-bit libunwind can not be installed at the same >>>> time. >>>> - # This will remove the 64-bit libunwind and install 32-bit >>>> version. >>>> - sudo apt-get install -y libunwind-dev:i386 >>>> +# Include packages needed by specific architectures. >>>> +if [ $TRAVIS_ARCH == amd64 ]; then >>>> + sudo apt-get install -y \ >>>> + libunbound-dev:i386 \ >>>> + gcc-multilib >>> >>> These packages are only needed for 32bit build, so you may just move >>> them to the command that installs 32bit version of libunwind. >>> And since you're not building 32bit with ppc64le, the code could >>> look >>> like: >>> >>> if [ "$M32" ]; then >>> # install 32 bit libs >>> fi >>> >>> if [ $TRAVIS_ARCH == ppc64le ]; then >>> # install ppc64le specific things. >>> fi >> >> Agreed. >> >>> >>>> + >>>> + if [ "$M32" ]; then >>>> + # 32-bit and 64-bit libunwind can not be installed at the >>>> + same >>>> time. >>>> + # This will remove the 64-bit libunwind and install 32-bit >>>> version. >>>> + sudo apt-get install -y libunwind-dev:i386 >>>> + fi >>>> + >>>> +elif [ $TRAVIS_ARCH == ppc64le ]; then >>>> + sudo apt-get install -y flex >>> >>> Why 'flex' is needed? >> >> I found that Flex was needed for kernel builds, but since I am not >> doing that yet I can remove it. >> >> Thanks for the review, I will post a V2 patch shortly. >> >>> >>> Best regards, Ilya Maximets. >> _______________________________________________ >> dev mailing list >> dev@openvswitch.org >> https://urldefense.proofpoint.com/v2/url?u=https-3A__mail.openvswitch >> .org_mailman_listinfo_ovs-2Ddev&d=DwIFAg&c=jf_iaSHvJObTbx-siA1ZOg&r=7 >> ndxyKjH9UrBD68Us2WP1wI4BwEBQbeAyz8i_vwCCaI&m=_QMOOwopXhBHpUCeEDvwONjc >> iJxP_WEksZXnZOBIFHM&s=W1XHbP-_Wjy0FfaSXS37Lx5F-tHW0oK9VRLwt5YpMro&e=
diff --git a/.travis.yml b/.travis.yml index 5676d9748..c99f26815 100644 --- a/.travis.yml +++ b/.travis.yml @@ -14,7 +14,6 @@ addons: apt: packages: - bc - - gcc-multilib - libssl-dev - llvm-dev - libjemalloc1 @@ -24,7 +23,6 @@ addons: - libelf-dev - selinux-policy-dev - libunbound-dev - - libunbound-dev:i386 - libunwind-dev before_install: ./.travis/${TRAVIS_OS_NAME}-prepare.sh @@ -50,6 +48,9 @@ matrix: - os: osx compiler: clang env: OPTS="--disable-ssl" + - os: linux-ppc64le + compiler: gcc + env: OPTS="--disable-ssl" script: ./.travis/${TRAVIS_OS_NAME}-build.sh $OPTS diff --git a/.travis/linux-prepare.sh b/.travis/linux-prepare.sh index e546d32cb..f3a9a6d44 100755 --- a/.travis/linux-prepare.sh +++ b/.travis/linux-prepare.sh @@ -15,8 +15,18 @@ cd .. pip install --disable-pip-version-check --user six flake8 hacking pip install --user --upgrade docutils -if [ "$M32" ]; then - # 32-bit and 64-bit libunwind can not be installed at the same time. - # This will remove the 64-bit libunwind and install 32-bit version. - sudo apt-get install -y libunwind-dev:i386 +# Include packages needed by specific architectures. +if [ $TRAVIS_ARCH == amd64 ]; then + sudo apt-get install -y \ + libunbound-dev:i386 \ + gcc-multilib + + if [ "$M32" ]; then + # 32-bit and 64-bit libunwind can not be installed at the same time. + # This will remove the 64-bit libunwind and install 32-bit version. + sudo apt-get install -y libunwind-dev:i386 + fi + +elif [ $TRAVIS_ARCH == ppc64le ]; then + sudo apt-get install -y flex fi
Add support for travis-ci ppc64le builds. - Updated matrix in .travis.yml to include a ppc64le build. - Added support to install packages needed by specific architectures. To keep the total build time at an acceptable level only a single build job is included in the matrix for ppc64le. A build report example can be found here [1] [0] http://travis-ci.org/ [1] https://travis-ci.org/djlwilder/ovs/builds/604098141 Signed-off-by: David Wilder <dwilder@us.ibm.com> --- .travis.yml | 5 +++-- .travis/linux-prepare.sh | 18 ++++++++++++++---- 2 files changed, 17 insertions(+), 6 deletions(-)