diff mbox series

[ovs-dev] ci: Fix handling of python packages.

Message ID 20210312163509.2578837-1-i.maximets@ovn.org
State Superseded
Headers show
Series [ovs-dev] ci: Fix handling of python packages. | expand

Commit Message

Ilya Maximets March 12, 2021, 4:35 p.m. UTC
GitHub Actions doesn't have python locations in PATH and different
runners might have different configuration for default python
location and versions.  For example, on some runners python2 might
be installed or not.

Missing PATH causes weird situations where during one run our scripts
can locate just installed flake8 and can't do that on a different run.

Also, we're mistakenly installing python2 version of flake8.
On runners that able to locate installed flake8 this causes breakage
of a flake8-check build target because our python scripts written for
python3.  And runners that can't locate flake8 works just fine and
job succeeds.

It's required to use actions/setup-python@v2 in order to have
predictable version of python installed and paths correctly configured.
Due to some bugs in GHA itself it doesn't set $HOME/.local/bin into
PATH, so we have to do that manually for now in order to use '--user'.

Fixes: ecdd790ecbff ("CI: Add github actions workflow.")
Reported-by: Numan Siddique <numans@ovn.org>
Signed-off-by: Ilya Maximets <i.maximets@ovn.org>
---

Will also work on similar patch for OVS.

 .ci/linux-prepare.sh       |  4 ++--
 .github/workflows/test.yml | 22 ++++++++++++++++++++--
 2 files changed, 22 insertions(+), 4 deletions(-)

Comments

Ilya Maximets March 12, 2021, 5:34 p.m. UTC | #1
On 3/12/21 5:35 PM, Ilya Maximets wrote:
> GitHub Actions doesn't have python locations in PATH and different
> runners might have different configuration for default python
> location and versions.  For example, on some runners python2 might
> be installed or not.
> 
> Missing PATH causes weird situations where during one run our scripts
> can locate just installed flake8 and can't do that on a different run.
> 
> Also, we're mistakenly installing python2 version of flake8.
> On runners that able to locate installed flake8 this causes breakage
> of a flake8-check build target because our python scripts written for
> python3.  And runners that can't locate flake8 works just fine and
> job succeeds.
> 
> It's required to use actions/setup-python@v2 in order to have
> predictable version of python installed and paths correctly configured.
> Due to some bugs in GHA itself it doesn't set $HOME/.local/bin into
> PATH, so we have to do that manually for now in order to use '--user'.
> 
> Fixes: ecdd790ecbff ("CI: Add github actions workflow.")
> Reported-by: Numan Siddique <numans@ovn.org>
> Signed-off-by: Ilya Maximets <i.maximets@ovn.org>
> ---
> 
> Will also work on similar patch for OVS.
> 
>  .ci/linux-prepare.sh       |  4 ++--
>  .github/workflows/test.yml | 22 ++++++++++++++++++++--
>  2 files changed, 22 insertions(+), 4 deletions(-)
> 
> diff --git a/.ci/linux-prepare.sh b/.ci/linux-prepare.sh
> index 0bb0ff096..55f419b63 100755
> --- a/.ci/linux-prepare.sh
> +++ b/.ci/linux-prepare.sh
> @@ -12,5 +12,5 @@ set -ev
>  git clone git://git.kernel.org/pub/scm/devel/sparse/sparse.git
>  cd sparse && make -j4 HAVE_LLVM= HAVE_SQLITE= install && cd ..
>  
> -pip install --disable-pip-version-check --user six flake8 hacking
> -pip install --user --upgrade docutils
> +pip3 install --disable-pip-version-check --user six flake8 hacking
> +pip3 install --upgrade --user docutils
> diff --git a/.github/workflows/test.yml b/.github/workflows/test.yml
> index 26a8edb8f..251672748 100644
> --- a/.github/workflows/test.yml
> +++ b/.github/workflows/test.yml
> @@ -97,11 +97,21 @@ jobs:
>        if:   matrix.m32 != ''
>        run:  sudo apt install -y ${{ env.m32_dependecies }}
>  
> +    - name: update PATH
> +      run:  |
> +        echo "$HOME/bin"        >> $GITHUB_PATH
> +        echo "$HOME/.local/bin" >> $GITHUB_PATH
> +
> +    - name: set up python
> +      uses: actions/setup-python@v2
> +      with:
> +        python-version: '3.x'

Ugh.  This change also makes invisible packages installed from
Ubuntu repositories.  Will move them to pip3 installation and
send v2.

> +
>      - name: prepare
>        run:  ./.ci/linux-prepare.sh
>  
>      - name: build
> -      run:  PATH="$PATH:$HOME/bin" ./.ci/linux-build.sh
> +      run:  ./.ci/linux-build.sh
>  
>      - name: copy logs on failure
>        if: failure() || cancelled()
> @@ -154,10 +164,18 @@ jobs:
>          ref: 'master'
>      - name: install dependencies
>        run:  brew install automake libtool
> +    - name: update PATH
> +      run:  |
> +        echo "$HOME/bin"        >> $GITHUB_PATH
> +        echo "$HOME/.local/bin" >> $GITHUB_PATH
> +    - name: set up python
> +      uses: actions/setup-python@v2
> +      with:
> +        python-version: '3.x'
>      - name: prepare
>        run:  ./.ci/osx-prepare.sh
>      - name: build
> -      run:  PATH="$PATH:$HOME/bin" ./.ci/osx-build.sh
> +      run:  ./.ci/osx-build.sh
>      - name: upload logs on failure
>        if: failure()
>        uses: actions/upload-artifact@v2
>
diff mbox series

Patch

diff --git a/.ci/linux-prepare.sh b/.ci/linux-prepare.sh
index 0bb0ff096..55f419b63 100755
--- a/.ci/linux-prepare.sh
+++ b/.ci/linux-prepare.sh
@@ -12,5 +12,5 @@  set -ev
 git clone git://git.kernel.org/pub/scm/devel/sparse/sparse.git
 cd sparse && make -j4 HAVE_LLVM= HAVE_SQLITE= install && cd ..
 
-pip install --disable-pip-version-check --user six flake8 hacking
-pip install --user --upgrade docutils
+pip3 install --disable-pip-version-check --user six flake8 hacking
+pip3 install --upgrade --user docutils
diff --git a/.github/workflows/test.yml b/.github/workflows/test.yml
index 26a8edb8f..251672748 100644
--- a/.github/workflows/test.yml
+++ b/.github/workflows/test.yml
@@ -97,11 +97,21 @@  jobs:
       if:   matrix.m32 != ''
       run:  sudo apt install -y ${{ env.m32_dependecies }}
 
+    - name: update PATH
+      run:  |
+        echo "$HOME/bin"        >> $GITHUB_PATH
+        echo "$HOME/.local/bin" >> $GITHUB_PATH
+
+    - name: set up python
+      uses: actions/setup-python@v2
+      with:
+        python-version: '3.x'
+
     - name: prepare
       run:  ./.ci/linux-prepare.sh
 
     - name: build
-      run:  PATH="$PATH:$HOME/bin" ./.ci/linux-build.sh
+      run:  ./.ci/linux-build.sh
 
     - name: copy logs on failure
       if: failure() || cancelled()
@@ -154,10 +164,18 @@  jobs:
         ref: 'master'
     - name: install dependencies
       run:  brew install automake libtool
+    - name: update PATH
+      run:  |
+        echo "$HOME/bin"        >> $GITHUB_PATH
+        echo "$HOME/.local/bin" >> $GITHUB_PATH
+    - name: set up python
+      uses: actions/setup-python@v2
+      with:
+        python-version: '3.x'
     - name: prepare
       run:  ./.ci/osx-prepare.sh
     - name: build
-      run:  PATH="$PATH:$HOME/bin" ./.ci/osx-build.sh
+      run:  ./.ci/osx-build.sh
     - name: upload logs on failure
       if: failure()
       uses: actions/upload-artifact@v2