| Message ID | 657cbef7-11ac-196c-01e2-aeb9744fe446@amsat.org |
|---|---|
| State | New |
| Headers | show |
| Series | gitlab-ci: amd64-opensuse-leap-container job failing | expand |
On Sun, Sep 26, 2021 at 07:23:56PM +0200, Philippe Mathieu-Daudé wrote: > Hi, > > FYI the OpenSUSE job is failing since few days, i.e.: > https://gitlab.com/qemu-project/qemu/-/jobs/1622345026 > > Retrieving repository 'Main Repository' metadata > [..........................................................error] > Repository 'Main Repository' is invalid. > > [repo-oss|http://download.opensuse.org/distribution/leap/15.2/repo/oss/] > Valid metadata not found at specified URL > History: > - Download (curl) error for > 'http://download.opensuse.org/distribution/leap/15.2/repo/oss/repodata/repomd.xml': > Error code: Curl error 56 > Error message: Recv failure: Connection reset by peer > - Can't provide /repodata/repomd.xml > Please check if the URIs defined for this repository are pointing to a > valid repository. > Warning: Skipping repository 'Main Repository' because of the above error. > > I tried to run 'zypper ref' with: It isn't confined to only SuSE. In libvirt we've had similar problems with several other jobs, though are suse jobs are the worst affected. GitLab have finally acknowledged it is an general infra issue affecting many things: https://status.gitlab.com/ https://gitlab.com/gitlab-com/gl-infra/production/-/issues/5590 > -- >8 -- > --- a/tests/docker/dockerfiles/opensuse-leap.docker > +++ b/tests/docker/dockerfiles/opensuse-leap.docker > @@ -109,5 +109,7 @@ ENV PACKAGES \ > zlib-devel > ENV QEMU_CONFIGURE_OPTS --python=/usr/bin/python3.6 > > -RUN zypper update -y && zypper --non-interactive install -y $PACKAGES > +RUN zypper refresh && \ > + zypper update -y && \ > + zypper --non-interactive install -y $PACKAGES > RUN rpm -q $PACKAGES | sort > /packages.txt > --- > > but no luck: https://gitlab.com/philmd/qemu/-/jobs/1623554962 > > Should we temporarily disable to job and its dependencies? Given it is believed to be a gitlab infra issue, rather than a problem of ours, or something we're using, I think best to wait a little longer to see if they get fix the infra. Regards, Daniel
On Mon, 2021-09-27 at 09:35 +0100, Daniel P. Berrangé wrote: > On Sun, Sep 26, 2021 at 07:23:56PM +0200, Philippe Mathieu-Daudé wrote: > > Hi, > > > > FYI the OpenSUSE job is failing since few days, i.e.: > > https://gitlab.com/qemu-project/qemu/-/jobs/1622345026 > > > > Retrieving repository 'Main Repository' metadata > > [..........................................................error] > > Repository 'Main Repository' is invalid. > > > > [repo- > > oss|http://download.opensuse.org/distribution/leap/15.2/repo/oss/] > > Valid metadata not found at specified URL > > History: > > - Download (curl) error for > > ' > > http://download.opensuse.org/distribution/leap/15.2/repo/oss/repodata/repomd.xml' > > : > > Error code: Curl error 56 > > Error message: Recv failure: Connection reset by peer > > - Can't provide /repodata/repomd.xml > > Please check if the URIs defined for this repository are pointing > > to a > > valid repository. > > Warning: Skipping repository 'Main Repository' because of the above > > error. > > > > I tried to run 'zypper ref' with: > > It isn't confined to only SuSE. In libvirt we've had similar problems > with several other jobs, though are suse jobs are the worst affected. > > GitLab have finally acknowledged it is an general infra issue affecting > many things: > > https://status.gitlab.com/ > https://gitlab.com/gitlab-com/gl-infra/production/-/issues/5590 > > > -- >8 -- > > --- a/tests/docker/dockerfiles/opensuse-leap.docker > > +++ b/tests/docker/dockerfiles/opensuse-leap.docker > > @@ -109,5 +109,7 @@ ENV PACKAGES \ > > zlib-devel > > ENV QEMU_CONFIGURE_OPTS --python=/usr/bin/python3.6 > > > > -RUN zypper update -y && zypper --non-interactive install -y > > $PACKAGES > > +RUN zypper refresh && \ > > + zypper update -y && \ > > + zypper --non-interactive install -y $PACKAGES > > RUN rpm -q $PACKAGES | sort > /packages.txt > > --- > > > > but no luck: https://gitlab.com/philmd/qemu/-/jobs/1623554962 > > > > Should we temporarily disable to job and its dependencies? > > Given it is believed to be a gitlab infra issue, rather than a problem > of ours, or something we're using, I think best to wait a little longer > to see if they get fix the infra. > agree, and I am also checking the status of it. for now the http://download.opensuse.org/distribution/leap/15.2 and the repo works. Will follow up it. Cheers, AL
On 9/27/21 10:35, Daniel P. Berrangé wrote: > On Sun, Sep 26, 2021 at 07:23:56PM +0200, Philippe Mathieu-Daudé wrote: >> Hi, >> >> FYI the OpenSUSE job is failing since few days, i.e.: >> https://gitlab.com/qemu-project/qemu/-/jobs/1622345026 > It isn't confined to only SuSE. In libvirt we've had similar problems > with several other jobs, though are suse jobs are the worst affected. > > GitLab have finally acknowledged it is an general infra issue affecting > many things: > > https://status.gitlab.com/ > https://gitlab.com/gitlab-com/gl-infra/production/-/issues/5590 >> Should we temporarily disable to job and its dependencies? > > Given it is believed to be a gitlab infra issue, rather than a problem > of ours, or something we're using, I think best to wait a little longer > to see if they get fix the infra. OK (I checked the status page during Saturday and Sunday morning and it was all green). Thanks for investigating, Phil.
On Mon, Sep 27, 2021 at 09:35:22AM +0100, Daniel P. Berrangé wrote: > On Sun, Sep 26, 2021 at 07:23:56PM +0200, Philippe Mathieu-Daudé wrote: > > Hi, > > > > FYI the OpenSUSE job is failing since few days, i.e.: > > https://gitlab.com/qemu-project/qemu/-/jobs/1622345026 > > > > Retrieving repository 'Main Repository' metadata > > [..........................................................error] > > Repository 'Main Repository' is invalid. > > > > [repo-oss|http://download.opensuse.org/distribution/leap/15.2/repo/oss/] > > Valid metadata not found at specified URL > > History: > > - Download (curl) error for > > 'http://download.opensuse.org/distribution/leap/15.2/repo/oss/repodata/repomd.xml': > > Error code: Curl error 56 > > Error message: Recv failure: Connection reset by peer > > - Can't provide /repodata/repomd.xml > > Please check if the URIs defined for this repository are pointing to a > > valid repository. > > Warning: Skipping repository 'Main Repository' because of the above error. > > > > I tried to run 'zypper ref' with: > > It isn't confined to only SuSE. In libvirt we've had similar problems > with several other jobs, though are suse jobs are the worst affected. > > GitLab have finally acknowledged it is an general infra issue affecting > many things: > > https://status.gitlab.com/ > https://gitlab.com/gitlab-com/gl-infra/production/-/issues/5590 Setting GitLab CI env var works around the problem temporarily: FF_NETWORK_PER_BUILD=true You can set it for all repos under a group eg https://gitlab.com/groups/qemu-project/-/settings/ci_cd or per repo eg https://gitlab.com/berrange/libvirt/-/settings/ci_cd Regards, Daniel
On 9/27/21 15:47, Daniel P. Berrangé wrote: > On Mon, Sep 27, 2021 at 09:35:22AM +0100, Daniel P. Berrangé wrote: >> On Sun, Sep 26, 2021 at 07:23:56PM +0200, Philippe Mathieu-Daudé wrote: >>> Hi, >>> >>> FYI the OpenSUSE job is failing since few days, i.e.: >>> https://gitlab.com/qemu-project/qemu/-/jobs/1622345026 >>> >>> Retrieving repository 'Main Repository' metadata >>> [..........................................................error] >>> Repository 'Main Repository' is invalid. >>> >>> [repo-oss|http://download.opensuse.org/distribution/leap/15.2/repo/oss/] >>> Valid metadata not found at specified URL >>> History: >>> - Download (curl) error for >>> 'http://download.opensuse.org/distribution/leap/15.2/repo/oss/repodata/repomd.xml': >>> Error code: Curl error 56 >>> Error message: Recv failure: Connection reset by peer >>> - Can't provide /repodata/repomd.xml >>> Please check if the URIs defined for this repository are pointing to a >>> valid repository. >>> Warning: Skipping repository 'Main Repository' because of the above error. >>> >>> I tried to run 'zypper ref' with: >> >> It isn't confined to only SuSE. In libvirt we've had similar problems >> with several other jobs, though are suse jobs are the worst affected. >> >> GitLab have finally acknowledged it is an general infra issue affecting >> many things: >> >> https://status.gitlab.com/ >> https://gitlab.com/gitlab-com/gl-infra/production/-/issues/5590 > > Setting GitLab CI env var works around the problem temporarily: > > FF_NETWORK_PER_BUILD=true Do you know if we need to recreate the pipeline? (It didn't work on already failing one, I'm going to test a freshly created one now). > You can set it for all repos under a group eg > > https://gitlab.com/groups/qemu-project/-/settings/ci_cd > > or per repo eg > > https://gitlab.com/berrange/libvirt/-/settings/ci_cd > > > Regards, > Daniel >
On Mon, Sep 27, 2021 at 04:35:04PM +0200, Philippe Mathieu-Daudé wrote: > On 9/27/21 15:47, Daniel P. Berrangé wrote: > > On Mon, Sep 27, 2021 at 09:35:22AM +0100, Daniel P. Berrangé wrote: > >> On Sun, Sep 26, 2021 at 07:23:56PM +0200, Philippe Mathieu-Daudé wrote: > >>> Hi, > >>> > >>> FYI the OpenSUSE job is failing since few days, i.e.: > >>> https://gitlab.com/qemu-project/qemu/-/jobs/1622345026 > >>> > >>> Retrieving repository 'Main Repository' metadata > >>> [..........................................................error] > >>> Repository 'Main Repository' is invalid. > >>> > >>> [repo-oss|http://download.opensuse.org/distribution/leap/15.2/repo/oss/] > >>> Valid metadata not found at specified URL > >>> History: > >>> - Download (curl) error for > >>> 'http://download.opensuse.org/distribution/leap/15.2/repo/oss/repodata/repomd.xml': > >>> Error code: Curl error 56 > >>> Error message: Recv failure: Connection reset by peer > >>> - Can't provide /repodata/repomd.xml > >>> Please check if the URIs defined for this repository are pointing to a > >>> valid repository. > >>> Warning: Skipping repository 'Main Repository' because of the above error. > >>> > >>> I tried to run 'zypper ref' with: > >> > >> It isn't confined to only SuSE. In libvirt we've had similar problems > >> with several other jobs, though are suse jobs are the worst affected. > >> > >> GitLab have finally acknowledged it is an general infra issue affecting > >> many things: > >> > >> https://status.gitlab.com/ > >> https://gitlab.com/gitlab-com/gl-infra/production/-/issues/5590 > > > > Setting GitLab CI env var works around the problem temporarily: > > > > FF_NETWORK_PER_BUILD=true > > Do you know if we need to recreate the pipeline? > (It didn't work on already failing one, I'm going to test > a freshly created one now). You shoudln't need to re-create the pipeline, just retry the job. If setting the variable in the web UI, make sure "Protect variable" is *not* set. Only the "master" branch is protected by default so other branches won't get protected variables set in their jobs. > > > You can set it for all repos under a group eg > > > > https://gitlab.com/groups/qemu-project/-/settings/ci_cd > > > > or per repo eg > > > > https://gitlab.com/berrange/libvirt/-/settings/ci_cd > > > > > > Regards, > > Daniel > > > Regards, Daniel
On Mon, Sep 27, 2021 at 09:35:22AM +0100, Daniel P. Berrangé wrote: > On Sun, Sep 26, 2021 at 07:23:56PM +0200, Philippe Mathieu-Daudé wrote: > > Hi, > > > > FYI the OpenSUSE job is failing since few days, i.e.: > > https://gitlab.com/qemu-project/qemu/-/jobs/1622345026 > > > > Retrieving repository 'Main Repository' metadata > > [..........................................................error] > > Repository 'Main Repository' is invalid. > > > > [repo-oss|http://download.opensuse.org/distribution/leap/15.2/repo/oss/] > > Valid metadata not found at specified URL > > History: > > - Download (curl) error for > > 'http://download.opensuse.org/distribution/leap/15.2/repo/oss/repodata/repomd.xml': > > Error code: Curl error 56 > > Error message: Recv failure: Connection reset by peer > > - Can't provide /repodata/repomd.xml > > Please check if the URIs defined for this repository are pointing to a > > valid repository. > > Warning: Skipping repository 'Main Repository' because of the above error. > > > > I tried to run 'zypper ref' with: > > It isn't confined to only SuSE. In libvirt we've had similar problems > with several other jobs, though are suse jobs are the worst affected. > > GitLab have finally acknowledged it is an general infra issue affecting > many things: > > https://status.gitlab.com/ > https://gitlab.com/gitlab-com/gl-infra/production/-/issues/5590 As it turned out, it wasn't gitlab's fault at all. Google cloud had some networking problems, and these were resolved a short while ago. So gitlab CI jobs sould be back to normal, and the FF_NETWORK_PER_BUILD env variable workaround should no longer be needed either. Regards, Daniel
--- a/tests/docker/dockerfiles/opensuse-leap.docker +++ b/tests/docker/dockerfiles/opensuse-leap.docker @@ -109,5 +109,7 @@ ENV PACKAGES \ zlib-devel ENV QEMU_CONFIGURE_OPTS --python=/usr/bin/python3.6 -RUN zypper update -y && zypper --non-interactive install -y $PACKAGES +RUN zypper refresh && \ + zypper update -y && \ + zypper --non-interactive install -y $PACKAGES RUN rpm -q $PACKAGES | sort > /packages.txt