diff mbox series

[ovs-dev,v2] CI: Add github actions workflow.

Message ID 20201126073517.178966-1-numans@ovn.org
State Changes Requested
Headers show
Series [ovs-dev,v2] CI: Add github actions workflow. | expand

Commit Message

Numan Siddique Nov. 26, 2020, 7:35 a.m. UTC 
From: Numan Siddique <numans@ovn.org>

This patch adds basic github actions to build and run OVN tests.
The test matrix is a slightly reduced version of current travis matrix.
We don't need to run OVN with multiple kernel versions.

This patch also removes the .travis.yml file as there is little value
in running the same tests in travis CI, given that travis-ci.org will
be turned into read-only mode. More information on this can be found
here [1].

We can further enhance the github actions to run ovn-kubernetes
end-to-end tests.

Thanks to Ilya, much of the github actions code is taken from Ilya's OVS
github action patch [2].

[1] - https://mail.openvswitch.org/pipermail/ovs-dev/2020-November/377773.html
[2] - https://patchwork.ozlabs.org/project/openvswitch/patch/20201125145753.534447-3-i.maximets@ovn.org/
Co-authored-by: Ilya Maximets <i.maximets@ovn.org>
Signed-off-by: Ilya Maximets <i.maximets@ovn.org>
Signed-off-by: Numan Siddique <numans@ovn.org>
---

Here is how it looks like:
  https://github.com/numansiddique/ovn/runs/1451852342

Antonio Ojea <aojea@redhat.com> has submitted a PR - https://github.com/ovn-org/ovn/pull/61
to run ovn-k8s end-to-end tests using github actions a while back. That PR
needs some polishing. I will be submitting that for review once it is
ready.


v1 -> v2
----
  * Addressed review comments from Ilya.

 {.travis => .ci}/linux-build.sh   | 18 ++++---
 {.travis => .ci}/linux-prepare.sh | 13 +++++
 {.travis => .ci}/osx-build.sh     |  0
 {.travis => .ci}/osx-prepare.sh   |  0
 .github/workflows/test.yml        | 83 +++++++++++++++++++++++++++++++
 .travis.yml                       | 47 -----------------
 Makefile.am                       | 10 ++--
 7 files changed, 111 insertions(+), 60 deletions(-)
 rename {.travis => .ci}/linux-build.sh (74%)
 rename {.travis => .ci}/linux-prepare.sh (55%)
 rename {.travis => .ci}/osx-build.sh (100%)
 rename {.travis => .ci}/osx-prepare.sh (100%)
 create mode 100644 .github/workflows/test.yml
 delete mode 100644 .travis.yml

Comments

Ilya Maximets Nov. 26, 2020, 1:23 p.m. UTC | #1 
On 11/26/20 8:35 AM, numans@ovn.org wrote:
> From: Numan Siddique <numans@ovn.org>
> 
> This patch adds basic github actions to build and run OVN tests.
> The test matrix is a slightly reduced version of current travis matrix.
> We don't need to run OVN with multiple kernel versions.
> 
> This patch also removes the .travis.yml file as there is little value
> in running the same tests in travis CI, given that travis-ci.org will
> be turned into read-only mode. More information on this can be found
> here [1].
> 
> We can further enhance the github actions to run ovn-kubernetes
> end-to-end tests.
> 
> Thanks to Ilya, much of the github actions code is taken from Ilya's OVS
> github action patch [2].
> 
> [1] - https://mail.openvswitch.org/pipermail/ovs-dev/2020-November/377773.html
> [2] - https://patchwork.ozlabs.org/project/openvswitch/patch/20201125145753.534447-3-i.maximets@ovn.org/
> Co-authored-by: Ilya Maximets <i.maximets@ovn.org>
> Signed-off-by: Ilya Maximets <i.maximets@ovn.org>
> Signed-off-by: Numan Siddique <numans@ovn.org>
> ---
> 
> Here is how it looks like:
>   https://github.com/numansiddique/ovn/runs/1451852342
> 
> Antonio Ojea <aojea@redhat.com> has submitted a PR - https://github.com/ovn-org/ovn/pull/61
> to run ovn-k8s end-to-end tests using github actions a while back. That PR
> needs some polishing. I will be submitting that for review once it is
> ready.
> 
> 
> v1 -> v2
> ----
>   * Addressed review comments from Ilya.
> 
>  {.travis => .ci}/linux-build.sh   | 18 ++++---
>  {.travis => .ci}/linux-prepare.sh | 13 +++++
>  {.travis => .ci}/osx-build.sh     |  0
>  {.travis => .ci}/osx-prepare.sh   |  0
>  .github/workflows/test.yml        | 83 +++++++++++++++++++++++++++++++
>  .travis.yml                       | 47 -----------------
>  Makefile.am                       | 10 ++--
>  7 files changed, 111 insertions(+), 60 deletions(-)
>  rename {.travis => .ci}/linux-build.sh (74%)
>  rename {.travis => .ci}/linux-prepare.sh (55%)
>  rename {.travis => .ci}/osx-build.sh (100%)
>  rename {.travis => .ci}/osx-prepare.sh (100%)
>  create mode 100644 .github/workflows/test.yml
>  delete mode 100644 .travis.yml
> 
> diff --git a/.travis/linux-build.sh b/.ci/linux-build.sh
> similarity index 74%
> rename from .travis/linux-build.sh
> rename to .ci/linux-build.sh
> index a8a561dc4e..0e9f87fa8b 100755
> --- a/.travis/linux-build.sh
> +++ b/.ci/linux-build.sh
> @@ -3,10 +3,9 @@
>  set -o errexit
>  set -x
>  
> -CFLAGS="-Werror"
> +CFLAGS=""
>  SPARSE_FLAGS=""
> -EXTRA_OPTS=""
> -TARGET="x86_64-native-linuxapp-gcc"
> +EXTRA_OPTS="--enable-Werror"
>  
>  function configure_ovs()
>  {
> @@ -24,16 +23,19 @@ function configure_ovn()
>      { cat config.log; exit 1; }
>  }
>  
> -OPTS="$EXTRA_OPTS $*"
> +save_OPTS="${OPTS} $*"
> +OPTS="${EXTRA_OPTS} ${save_OPTS}"
>  
>  if [ "$CC" = "clang" ]; then
>      export OVS_CFLAGS="$CFLAGS -Wno-error=unused-command-line-argument"
> -elif [[ $BUILD_ENV =~ "-m32" ]]; then
> -    # Disable sparse for 32bit builds on 64bit machine
> -    export OVS_CFLAGS="$CFLAGS $BUILD_ENV"
> +elif [ "$M32" ]; then
> +    # Not using sparse for 32bit builds on 64bit machine.
> +    # Adding m32 flag directly to CC to avoid any posiible issues with API/ABI
> +    # difference on 'configure' and 'make' stages.
> +    export CC="$CC -m32"
>  else
>      OPTS="$OPTS --enable-sparse"
> -    export OVS_CFLAGS="$CFLAGS $BUILD_ENV $SPARSE_FLAGS"
> +    export OVS_CFLAGS="$CFLAGS $SPARSE_FLAGS"
>  fi
>  
>  if [ "$TESTSUITE" ]; then
> diff --git a/.travis/linux-prepare.sh b/.ci/linux-prepare.sh
> similarity index 55%
> rename from .travis/linux-prepare.sh
> rename to .ci/linux-prepare.sh
> index 0bb0ff096e..4e94312f36 100755
> --- a/.travis/linux-prepare.sh
> +++ b/.ci/linux-prepare.sh
> @@ -14,3 +14,16 @@ cd sparse && make -j4 HAVE_LLVM= HAVE_SQLITE= install && cd ..
>  
>  pip install --disable-pip-version-check --user six flake8 hacking
>  pip install --user --upgrade docutils
> +
> +if [ "$M32" ]; then
> +    # Installing 32-bit libraries.
> +    pkgs="gcc-multilib"
> +    if [ -z "$GITHUB_WORKFLOW" ]; then

Since you're removing Travis support, this 'if' condition will always
be false, i.e. 32-bit versions of libunwind and libunbound will never
be installed, so you could just remove it.  You may also just add
'gcc-multilib' to the list of dependencies and avoid changing
linux-prepare.sh.

> +        # 32-bit and 64-bit libunwind can not be installed at the same time.
> +        # This will remove the 64-bit libunwind and install 32-bit version.
> +        # GitHub Actions doesn't have 32-bit versions of these libs.
> +        pkgs=$pkgs" libunwind-dev:i386 libunbound-dev:i386"
> +    fi
> +
> +    sudo apt-get install -y $pkgs
> +fi
> diff --git a/.travis/osx-build.sh b/.ci/osx-build.sh
> similarity index 100%
> rename from .travis/osx-build.sh
> rename to .ci/osx-build.sh
> diff --git a/.travis/osx-prepare.sh b/.ci/osx-prepare.sh
> similarity index 100%
> rename from .travis/osx-prepare.sh
> rename to .ci/osx-prepare.sh
> diff --git a/.github/workflows/test.yml b/.github/workflows/test.yml
> new file mode 100644
> index 0000000000..38389d5510
> --- /dev/null
> +++ b/.github/workflows/test.yml
> @@ -0,0 +1,83 @@
> +name: Build and Test
> +
> +on: [push, pull_request]
> +
> +jobs:
> +  build-linux:
> +    env:
> +      dependencies: |
> +        automake libtool gcc bc libjemalloc1 libjemalloc-dev    \
> +        libssl-dev llvm-dev libelf-dev libnuma-dev libpcap-dev  \
> +        python3-openssl python3-pip python3-sphinx              \
> +        selinux-policy-dev
> +      CC:          ${{ matrix.compiler }}
> +      LIBS:        ${{ matrix.libs }}
> +      M32:         ${{ matrix.m32 }}
> +      OPTS:        ${{ matrix.opts }}
> +      TESTSUITE:   ${{ matrix.testsuite }}
> +
> +    name: linux ${{ join(matrix.*, ' ') }}
> +    runs-on: ubuntu-18.04
> +
> +    strategy:
> +      fail-fast: false
> +      matrix:
> +        include:
> +          - compiler:     gcc
> +            opts:         --disable-ssl
> +          - compiler:     clang
> +            opts:         --disable-ssl
> +
> +          - compiler:     gcc
> +            testsuite:    test
> +          - compiler:     clang
> +            testsuite:    test
> +
> +          - compiler:     gcc
> +            testsuite:    test
> +            libs:         -ljemalloc
> +          - compiler:     clang
> +            testsuite:    test
> +            libs:         -ljemalloc
> +
> +          - compiler:     gcc
> +            m32:          m32
> +            opts:         --disable-ssl
> +
> +    steps:
> +    - name: checkout
> +      uses: actions/checkout@v2
> +
> +    - name: install required dependencies
> +      run:  sudo apt install -y ${{ env.dependencies }}
> +
> +    - name: install libunbound libunwind
> +      if:   matrix.m32 == ''
> +      run:  sudo apt install -y libunbound-dev libunwind-dev
> +
> +    - name: prepare
> +      run:  ./.ci/linux-prepare.sh
> +
> +    - name: build
> +      run:  PATH="$PATH:$HOME/bin" ./.ci/linux-build.sh

It might be beneficial for debugging to have logs and testsuite
artifacts uploaded.  It might be enough to just copy 'copy logs on failure'
and 'upload logs on failure' steps from v3 of my OVS patches:
https://patchwork.ozlabs.org/project/openvswitch/patch/20201126121105.652264-2-i.maximets@ovn.org/

> +
> +  build-osx:
> +    env:
> +      CC:    clang
> +      OPTS:  --disable-ssl
> +
> +    name:    osx clang --disable-ssl
> +    runs-on: macos-latest
> +
> +    strategy:
> +      fail-fast: false
> +
> +    steps:
> +    - name: checkout
> +      uses: actions/checkout@v2
> +    - name: install dependencies
> +      run:  brew install automake libtool
> +    - name: prepare
> +      run:  ./.ci/osx-prepare.sh
> +    - name: build
> +      run:  PATH="$PATH:$HOME/bin" ./.ci/osx-build.sh
> diff --git a/.travis.yml b/.travis.yml
> deleted file mode 100644
> index cd853fddd9..0000000000
> --- a/.travis.yml
> +++ /dev/null
> @@ -1,47 +0,0 @@
> -language: c
> -compiler:
> -  - gcc
> -  - clang
> -
> -os:
> -  - linux
> -
> -addons:
> -  apt:
> -    packages:
> -      - bc
> -      - gcc-multilib
> -      - libssl-dev
> -      - llvm-dev
> -      - libjemalloc1
> -      - libjemalloc-dev
> -      - libnuma-dev
> -      - python-sphinx
> -      - libelf-dev
> -      - selinux-policy-dev
> -      - libunbound-dev
> -      - libunbound-dev:i386
> -
> -before_install: ./.travis/${TRAVIS_OS_NAME}-prepare.sh
> -
> -before_script: export PATH=$PATH:$HOME/bin
> -
> -env:
> -  - OPTS="--disable-ssl"
> -  - TESTSUITE=1 KERNEL=4.18.20
> -  - TESTSUITE=1 OPTS="--enable-shared"
> -  - BUILD_ENV="-m32" OPTS="--disable-ssl"
> -  - TESTSUITE=1 LIBS=-ljemalloc
> -
> -matrix:
> -  include:
> -    - os: osx
> -      compiler: clang
> -      env: OPTS="--disable-ssl"
> -
> -script: ./.travis/${TRAVIS_OS_NAME}-build.sh $OPTS
> -
> -notifications:
> -  email:
> -    recipients:
> -      - ovs-build@openvswitch.org
> diff --git a/Makefile.am b/Makefile.am
> index 221ca61be7..7ce3d27e46 100644
> --- a/Makefile.am
> +++ b/Makefile.am
> @@ -84,11 +84,11 @@ EXTRA_DIST = \
>  	README.rst \
>  	NOTICE \
>  	.cirrus.yml \
> -	.travis.yml \
> -	.travis/linux-build.sh \
> -	.travis/linux-prepare.sh \
> -	.travis/osx-build.sh \
> -	.travis/osx-prepare.sh \
> +	.ci/linux-build.sh \
> +	.ci/linux-prepare.sh \
> +	.ci/osx-build.sh \
> +	.ci/osx-prepare.sh \
> +	.github/workflows/test.yml \
>  	boot.sh \
>  	$(MAN_FRAGMENTS) \
>  	$(MAN_ROOTS) \
>
diff mbox series

Patch

diff --git a/.travis/linux-build.sh b/.ci/linux-build.sh
similarity index 74%
rename from .travis/linux-build.sh
rename to .ci/linux-build.sh
index a8a561dc4e..0e9f87fa8b 100755
--- a/.travis/linux-build.sh
+++ b/.ci/linux-build.sh
@@ -3,10 +3,9 @@ 
 set -o errexit
 set -x
 
-CFLAGS="-Werror"
+CFLAGS=""
 SPARSE_FLAGS=""
-EXTRA_OPTS=""
-TARGET="x86_64-native-linuxapp-gcc"
+EXTRA_OPTS="--enable-Werror"
 
 function configure_ovs()
 {
@@ -24,16 +23,19 @@  function configure_ovn()
     { cat config.log; exit 1; }
 }
 
-OPTS="$EXTRA_OPTS $*"
+save_OPTS="${OPTS} $*"
+OPTS="${EXTRA_OPTS} ${save_OPTS}"
 
 if [ "$CC" = "clang" ]; then
     export OVS_CFLAGS="$CFLAGS -Wno-error=unused-command-line-argument"
-elif [[ $BUILD_ENV =~ "-m32" ]]; then
-    # Disable sparse for 32bit builds on 64bit machine
-    export OVS_CFLAGS="$CFLAGS $BUILD_ENV"
+elif [ "$M32" ]; then
+    # Not using sparse for 32bit builds on 64bit machine.
+    # Adding m32 flag directly to CC to avoid any posiible issues with API/ABI
+    # difference on 'configure' and 'make' stages.
+    export CC="$CC -m32"
 else
     OPTS="$OPTS --enable-sparse"
-    export OVS_CFLAGS="$CFLAGS $BUILD_ENV $SPARSE_FLAGS"
+    export OVS_CFLAGS="$CFLAGS $SPARSE_FLAGS"
 fi
 
 if [ "$TESTSUITE" ]; then
diff --git a/.travis/linux-prepare.sh b/.ci/linux-prepare.sh
similarity index 55%
rename from .travis/linux-prepare.sh
rename to .ci/linux-prepare.sh
index 0bb0ff096e..4e94312f36 100755
--- a/.travis/linux-prepare.sh
+++ b/.ci/linux-prepare.sh
@@ -14,3 +14,16 @@  cd sparse && make -j4 HAVE_LLVM= HAVE_SQLITE= install && cd ..
 
 pip install --disable-pip-version-check --user six flake8 hacking
 pip install --user --upgrade docutils
+
+if [ "$M32" ]; then
+    # Installing 32-bit libraries.
+    pkgs="gcc-multilib"
+    if [ -z "$GITHUB_WORKFLOW" ]; then
+        # 32-bit and 64-bit libunwind can not be installed at the same time.
+        # This will remove the 64-bit libunwind and install 32-bit version.
+        # GitHub Actions doesn't have 32-bit versions of these libs.
+        pkgs=$pkgs" libunwind-dev:i386 libunbound-dev:i386"
+    fi
+
+    sudo apt-get install -y $pkgs
+fi
diff --git a/.travis/osx-build.sh b/.ci/osx-build.sh
similarity index 100%
rename from .travis/osx-build.sh
rename to .ci/osx-build.sh
diff --git a/.travis/osx-prepare.sh b/.ci/osx-prepare.sh
similarity index 100%
rename from .travis/osx-prepare.sh
rename to .ci/osx-prepare.sh
diff --git a/.github/workflows/test.yml b/.github/workflows/test.yml
new file mode 100644
index 0000000000..38389d5510
--- /dev/null
+++ b/.github/workflows/test.yml
@@ -0,0 +1,83 @@ 
+name: Build and Test
+
+on: [push, pull_request]
+
+jobs:
+  build-linux:
+    env:
+      dependencies: |
+        automake libtool gcc bc libjemalloc1 libjemalloc-dev    \
+        libssl-dev llvm-dev libelf-dev libnuma-dev libpcap-dev  \
+        python3-openssl python3-pip python3-sphinx              \
+        selinux-policy-dev
+      CC:          ${{ matrix.compiler }}
+      LIBS:        ${{ matrix.libs }}
+      M32:         ${{ matrix.m32 }}
+      OPTS:        ${{ matrix.opts }}
+      TESTSUITE:   ${{ matrix.testsuite }}
+
+    name: linux ${{ join(matrix.*, ' ') }}
+    runs-on: ubuntu-18.04
+
+    strategy:
+      fail-fast: false
+      matrix:
+        include:
+          - compiler:     gcc
+            opts:         --disable-ssl
+          - compiler:     clang
+            opts:         --disable-ssl
+
+          - compiler:     gcc
+            testsuite:    test
+          - compiler:     clang
+            testsuite:    test
+
+          - compiler:     gcc
+            testsuite:    test
+            libs:         -ljemalloc
+          - compiler:     clang
+            testsuite:    test
+            libs:         -ljemalloc
+
+          - compiler:     gcc
+            m32:          m32
+            opts:         --disable-ssl
+
+    steps:
+    - name: checkout
+      uses: actions/checkout@v2
+
+    - name: install required dependencies
+      run:  sudo apt install -y ${{ env.dependencies }}
+
+    - name: install libunbound libunwind
+      if:   matrix.m32 == ''
+      run:  sudo apt install -y libunbound-dev libunwind-dev
+
+    - name: prepare
+      run:  ./.ci/linux-prepare.sh
+
+    - name: build
+      run:  PATH="$PATH:$HOME/bin" ./.ci/linux-build.sh
+
+  build-osx:
+    env:
+      CC:    clang
+      OPTS:  --disable-ssl
+
+    name:    osx clang --disable-ssl
+    runs-on: macos-latest
+
+    strategy:
+      fail-fast: false
+
+    steps:
+    - name: checkout
+      uses: actions/checkout@v2
+    - name: install dependencies
+      run:  brew install automake libtool
+    - name: prepare
+      run:  ./.ci/osx-prepare.sh
+    - name: build
+      run:  PATH="$PATH:$HOME/bin" ./.ci/osx-build.sh
diff --git a/.travis.yml b/.travis.yml
deleted file mode 100644
index cd853fddd9..0000000000
--- a/.travis.yml
+++ /dev/null
@@ -1,47 +0,0 @@ 
-language: c
-compiler:
-  - gcc
-  - clang
-
-os:
-  - linux
-
-addons:
-  apt:
-    packages:
-      - bc
-      - gcc-multilib
-      - libssl-dev
-      - llvm-dev
-      - libjemalloc1
-      - libjemalloc-dev
-      - libnuma-dev
-      - python-sphinx
-      - libelf-dev
-      - selinux-policy-dev
-      - libunbound-dev
-      - libunbound-dev:i386
-
-before_install: ./.travis/${TRAVIS_OS_NAME}-prepare.sh
-
-before_script: export PATH=$PATH:$HOME/bin
-
-env:
-  - OPTS="--disable-ssl"
-  - TESTSUITE=1 KERNEL=4.18.20
-  - TESTSUITE=1 OPTS="--enable-shared"
-  - BUILD_ENV="-m32" OPTS="--disable-ssl"
-  - TESTSUITE=1 LIBS=-ljemalloc
-
-matrix:
-  include:
-    - os: osx
-      compiler: clang
-      env: OPTS="--disable-ssl"
-
-script: ./.travis/${TRAVIS_OS_NAME}-build.sh $OPTS
-
-notifications:
-  email:
-    recipients:
-      - ovs-build@openvswitch.org
diff --git a/Makefile.am b/Makefile.am
index 221ca61be7..7ce3d27e46 100644
--- a/Makefile.am
+++ b/Makefile.am
@@ -84,11 +84,11 @@  EXTRA_DIST = \
 	README.rst \
 	NOTICE \
 	.cirrus.yml \
-	.travis.yml \
-	.travis/linux-build.sh \
-	.travis/linux-prepare.sh \
-	.travis/osx-build.sh \
-	.travis/osx-prepare.sh \
+	.ci/linux-build.sh \
+	.ci/linux-prepare.sh \
+	.ci/osx-build.sh \
+	.ci/osx-prepare.sh \
+	.github/workflows/test.yml \
 	boot.sh \
 	$(MAN_FRAGMENTS) \
 	$(MAN_ROOTS) \