Message ID | 20210312163509.2578837-1-i.maximets@ovn.org |
---|---|
State | Superseded |
Headers | show |
Series | [ovs-dev] ci: Fix handling of python packages. | expand |
On 3/12/21 5:35 PM, Ilya Maximets wrote: > GitHub Actions doesn't have python locations in PATH and different > runners might have different configuration for default python > location and versions. For example, on some runners python2 might > be installed or not. > > Missing PATH causes weird situations where during one run our scripts > can locate just installed flake8 and can't do that on a different run. > > Also, we're mistakenly installing python2 version of flake8. > On runners that able to locate installed flake8 this causes breakage > of a flake8-check build target because our python scripts written for > python3. And runners that can't locate flake8 works just fine and > job succeeds. > > It's required to use actions/setup-python@v2 in order to have > predictable version of python installed and paths correctly configured. > Due to some bugs in GHA itself it doesn't set $HOME/.local/bin into > PATH, so we have to do that manually for now in order to use '--user'. > > Fixes: ecdd790ecbff ("CI: Add github actions workflow.") > Reported-by: Numan Siddique <numans@ovn.org> > Signed-off-by: Ilya Maximets <i.maximets@ovn.org> > --- > > Will also work on similar patch for OVS. > > .ci/linux-prepare.sh | 4 ++-- > .github/workflows/test.yml | 22 ++++++++++++++++++++-- > 2 files changed, 22 insertions(+), 4 deletions(-) > > diff --git a/.ci/linux-prepare.sh b/.ci/linux-prepare.sh > index 0bb0ff096..55f419b63 100755 > --- a/.ci/linux-prepare.sh > +++ b/.ci/linux-prepare.sh > @@ -12,5 +12,5 @@ set -ev > git clone git://git.kernel.org/pub/scm/devel/sparse/sparse.git > cd sparse && make -j4 HAVE_LLVM= HAVE_SQLITE= install && cd .. > > -pip install --disable-pip-version-check --user six flake8 hacking > -pip install --user --upgrade docutils > +pip3 install --disable-pip-version-check --user six flake8 hacking > +pip3 install --upgrade --user docutils > diff --git a/.github/workflows/test.yml b/.github/workflows/test.yml > index 26a8edb8f..251672748 100644 > --- a/.github/workflows/test.yml > +++ b/.github/workflows/test.yml > @@ -97,11 +97,21 @@ jobs: > if: matrix.m32 != '' > run: sudo apt install -y ${{ env.m32_dependecies }} > > + - name: update PATH > + run: | > + echo "$HOME/bin" >> $GITHUB_PATH > + echo "$HOME/.local/bin" >> $GITHUB_PATH > + > + - name: set up python > + uses: actions/setup-python@v2 > + with: > + python-version: '3.x' Ugh. This change also makes invisible packages installed from Ubuntu repositories. Will move them to pip3 installation and send v2. > + > - name: prepare > run: ./.ci/linux-prepare.sh > > - name: build > - run: PATH="$PATH:$HOME/bin" ./.ci/linux-build.sh > + run: ./.ci/linux-build.sh > > - name: copy logs on failure > if: failure() || cancelled() > @@ -154,10 +164,18 @@ jobs: > ref: 'master' > - name: install dependencies > run: brew install automake libtool > + - name: update PATH > + run: | > + echo "$HOME/bin" >> $GITHUB_PATH > + echo "$HOME/.local/bin" >> $GITHUB_PATH > + - name: set up python > + uses: actions/setup-python@v2 > + with: > + python-version: '3.x' > - name: prepare > run: ./.ci/osx-prepare.sh > - name: build > - run: PATH="$PATH:$HOME/bin" ./.ci/osx-build.sh > + run: ./.ci/osx-build.sh > - name: upload logs on failure > if: failure() > uses: actions/upload-artifact@v2 >
diff --git a/.ci/linux-prepare.sh b/.ci/linux-prepare.sh index 0bb0ff096..55f419b63 100755 --- a/.ci/linux-prepare.sh +++ b/.ci/linux-prepare.sh @@ -12,5 +12,5 @@ set -ev git clone git://git.kernel.org/pub/scm/devel/sparse/sparse.git cd sparse && make -j4 HAVE_LLVM= HAVE_SQLITE= install && cd .. -pip install --disable-pip-version-check --user six flake8 hacking -pip install --user --upgrade docutils +pip3 install --disable-pip-version-check --user six flake8 hacking +pip3 install --upgrade --user docutils diff --git a/.github/workflows/test.yml b/.github/workflows/test.yml index 26a8edb8f..251672748 100644 --- a/.github/workflows/test.yml +++ b/.github/workflows/test.yml @@ -97,11 +97,21 @@ jobs: if: matrix.m32 != '' run: sudo apt install -y ${{ env.m32_dependecies }} + - name: update PATH + run: | + echo "$HOME/bin" >> $GITHUB_PATH + echo "$HOME/.local/bin" >> $GITHUB_PATH + + - name: set up python + uses: actions/setup-python@v2 + with: + python-version: '3.x' + - name: prepare run: ./.ci/linux-prepare.sh - name: build - run: PATH="$PATH:$HOME/bin" ./.ci/linux-build.sh + run: ./.ci/linux-build.sh - name: copy logs on failure if: failure() || cancelled() @@ -154,10 +164,18 @@ jobs: ref: 'master' - name: install dependencies run: brew install automake libtool + - name: update PATH + run: | + echo "$HOME/bin" >> $GITHUB_PATH + echo "$HOME/.local/bin" >> $GITHUB_PATH + - name: set up python + uses: actions/setup-python@v2 + with: + python-version: '3.x' - name: prepare run: ./.ci/osx-prepare.sh - name: build - run: PATH="$PATH:$HOME/bin" ./.ci/osx-build.sh + run: ./.ci/osx-build.sh - name: upload logs on failure if: failure() uses: actions/upload-artifact@v2
GitHub Actions doesn't have python locations in PATH and different runners might have different configuration for default python location and versions. For example, on some runners python2 might be installed or not. Missing PATH causes weird situations where during one run our scripts can locate just installed flake8 and can't do that on a different run. Also, we're mistakenly installing python2 version of flake8. On runners that able to locate installed flake8 this causes breakage of a flake8-check build target because our python scripts written for python3. And runners that can't locate flake8 works just fine and job succeeds. It's required to use actions/setup-python@v2 in order to have predictable version of python installed and paths correctly configured. Due to some bugs in GHA itself it doesn't set $HOME/.local/bin into PATH, so we have to do that manually for now in order to use '--user'. Fixes: ecdd790ecbff ("CI: Add github actions workflow.") Reported-by: Numan Siddique <numans@ovn.org> Signed-off-by: Ilya Maximets <i.maximets@ovn.org> --- Will also work on similar patch for OVS. .ci/linux-prepare.sh | 4 ++-- .github/workflows/test.yml | 22 ++++++++++++++++++++-- 2 files changed, 22 insertions(+), 4 deletions(-)