[PULL,0/1] Require Python >= 3.5 to build QEMU
mbox

Message ID 20191025203427.20181-1-ehabkost@redhat.com
State New
Headers show

Pull-request

git://github.com/ehabkost/qemu.git tags/python-next-pull-request

Message

Eduardo Habkost Oct. 25, 2019, 8:34 p.m. UTC
The following changes since commit 03bf012e523ecdf047ac56b2057950247256064d:

  Merge remote-tracking branch 'remotes/kevin/tags/for-upstream' into staging (2019-10-25 14:59:53 +0100)

are available in the Git repository at:

  git://github.com/ehabkost/qemu.git tags/python-next-pull-request

for you to fetch changes up to d24e417866f85229de1b75bc5c0a1d942451a842:

  configure: Require Python >= 3.5 (2019-10-25 16:34:57 -0300)

----------------------------------------------------------------
Require Python >= 3.5 to build QEMU

----------------------------------------------------------------

Eduardo Habkost (1):
  configure: Require Python >= 3.5

 configure              | 18 ++++--------------
 tests/Makefile.include |  5 -----
 2 files changed, 4 insertions(+), 19 deletions(-)

Comments

Peter Maydell Oct. 31, 2019, 8:12 a.m. UTC | #1
On Fri, 25 Oct 2019 at 21:34, Eduardo Habkost <ehabkost@redhat.com> wrote:
>
> The following changes since commit 03bf012e523ecdf047ac56b2057950247256064d:
>
>   Merge remote-tracking branch 'remotes/kevin/tags/for-upstream' into staging (2019-10-25 14:59:53 +0100)
>
> are available in the Git repository at:
>
>   git://github.com/ehabkost/qemu.git tags/python-next-pull-request
>
> for you to fetch changes up to d24e417866f85229de1b75bc5c0a1d942451a842:
>
>   configure: Require Python >= 3.5 (2019-10-25 16:34:57 -0300)
>
> ----------------------------------------------------------------
> Require Python >= 3.5 to build QEMU
>
> ----------------------------------------------------------------

I can't apply this until we've fixed the tests/vm netbsd setup to
not use Python 2.

Have you tried a test run with Travis/etc/etc to check that none of
those CI configs need updating to have python3 available ?

thanks
-- PMM
Eduardo Habkost Nov. 5, 2019, 7:57 p.m. UTC | #2
On Thu, Oct 31, 2019 at 08:12:01AM +0000, Peter Maydell wrote:
> On Fri, 25 Oct 2019 at 21:34, Eduardo Habkost <ehabkost@redhat.com> wrote:
> >
> > The following changes since commit 03bf012e523ecdf047ac56b2057950247256064d:
> >
> >   Merge remote-tracking branch 'remotes/kevin/tags/for-upstream' into staging (2019-10-25 14:59:53 +0100)
> >
> > are available in the Git repository at:
> >
> >   git://github.com/ehabkost/qemu.git tags/python-next-pull-request
> >
> > for you to fetch changes up to d24e417866f85229de1b75bc5c0a1d942451a842:
> >
> >   configure: Require Python >= 3.5 (2019-10-25 16:34:57 -0300)
> >
> > ----------------------------------------------------------------
> > Require Python >= 3.5 to build QEMU
> >
> > ----------------------------------------------------------------
> 
> I can't apply this until we've fixed the tests/vm netbsd setup to
> not use Python 2.

Fixing tests/vm/netbsd is being tricky.  It looks like the
configure patch will have to wait until after QEMU 4.2.0.  :(

> 
> Have you tried a test run with Travis/etc/etc to check that none of
> those CI configs need updating to have python3 available ?

I have tested this pull request on Shippable, and I will take a
look at Travis.  I'd appreciate help from the CI system
maintainers (CCed) for the rest, as I don't have accounts in all
our CI systems.

Do we expect maintainers to test their pull requests in all CI
systems listed at the QEMU wiki[1]?  Do we have an official list
of CI systems that you consider to be pull request blockers?

[1] https://wiki.qemu.org/Testing#Continuous_Integration
Peter Maydell Nov. 5, 2019, 8:10 p.m. UTC | #3
On Tue, 5 Nov 2019 at 19:57, Eduardo Habkost <ehabkost@redhat.com> wrote:
> Fixing tests/vm/netbsd is being tricky.  It looks like the
> configure patch will have to wait until after QEMU 4.2.0.  :(

I think that makes sense at this point in the release cycle, yes.

> > Have you tried a test run with Travis/etc/etc to check that none of
> > those CI configs need updating to have python3 available ?
>
> I have tested this pull request on Shippable, and I will take a
> look at Travis.  I'd appreciate help from the CI system
> maintainers (CCed) for the rest, as I don't have accounts in all
> our CI systems.
>
> Do we expect maintainers to test their pull requests in all CI
> systems listed at the QEMU wiki[1]?  Do we have an official list
> of CI systems that you consider to be pull request blockers?

No, I don't in general expect people to test on all these CI
systems. I ask in this specific case because trying to
move to python-3-only seems like a change that's quite
likely to break the CI setups unless we've already checked
that all their configs will be pulling in a python 3 already.

thanks
-- PMM
Alex Bennée Nov. 5, 2019, 8:25 p.m. UTC | #4
Eduardo Habkost <ehabkost@redhat.com> writes:

> On Thu, Oct 31, 2019 at 08:12:01AM +0000, Peter Maydell wrote:
>> On Fri, 25 Oct 2019 at 21:34, Eduardo Habkost <ehabkost@redhat.com> wrote:
>> >
>> > The following changes since commit 03bf012e523ecdf047ac56b2057950247256064d:
>> >
>> >   Merge remote-tracking branch 'remotes/kevin/tags/for-upstream' into staging (2019-10-25 14:59:53 +0100)
>> >
>> > are available in the Git repository at:
>> >
>> >   git://github.com/ehabkost/qemu.git tags/python-next-pull-request
>> >
>> > for you to fetch changes up to d24e417866f85229de1b75bc5c0a1d942451a842:
>> >
>> >   configure: Require Python >= 3.5 (2019-10-25 16:34:57 -0300)
>> >
>> > ----------------------------------------------------------------
>> > Require Python >= 3.5 to build QEMU
>> >
>> > ----------------------------------------------------------------
>>
>> I can't apply this until we've fixed the tests/vm netbsd setup to
>> not use Python 2.
>
> Fixing tests/vm/netbsd is being tricky.  It looks like the
> configure patch will have to wait until after QEMU 4.2.0.  :(

I've posted fixes for the netbsd serial install but there are still
problems with the tests including what looks like a fairly serious
failure in the async code.

>
>>
>> Have you tried a test run with Travis/etc/etc to check that none of
>> those CI configs need updating to have python3 available ?
>
> I have tested this pull request on Shippable, and I will take a
> look at Travis.  I'd appreciate help from the CI system
> maintainers (CCed) for the rest, as I don't have accounts in all
> our CI systems.

Setting up accounts on the others doesn't take long. I use the
CustomCIStatus template to instantiate all the buttons for my various
maintainer branches on the wiki, e.g.:

  {{CustomCIStatus|user=stsquad|repo=qemu|branch=testing/next|ship_proj=5885eac43b653a0f00fa97f5}}

which means I just have to glance at the button state rather than going
through each individual CI's status pages.

> Do we expect maintainers to test their pull requests in all CI
> systems listed at the QEMU wiki[1]?  Do we have an official list
> of CI systems that you consider to be pull request blockers?

Well they all catch various things but none of them catch all the things
Peter's PR processing does. Historically Travis has been allowed to
slide because of test instability and timeouts. Having said that last I
checked everything was green so breaking any of the main CIs
(Travis/Shippable/Cirrus/Gitlab) indicates there is a problem that needs
to be fixed.

>
> [1] https://wiki.qemu.org/Testing#Continuous_Integration


--
Alex Bennée
Eduardo Habkost Nov. 5, 2019, 8:36 p.m. UTC | #5
On Tue, Nov 05, 2019 at 08:25:03PM +0000, Alex Bennée wrote:
> 
> Eduardo Habkost <ehabkost@redhat.com> writes:
> 
> > On Thu, Oct 31, 2019 at 08:12:01AM +0000, Peter Maydell wrote:
> >> On Fri, 25 Oct 2019 at 21:34, Eduardo Habkost <ehabkost@redhat.com> wrote:
> >> >
> >> > The following changes since commit 03bf012e523ecdf047ac56b2057950247256064d:
> >> >
> >> >   Merge remote-tracking branch 'remotes/kevin/tags/for-upstream' into staging (2019-10-25 14:59:53 +0100)
> >> >
> >> > are available in the Git repository at:
> >> >
> >> >   git://github.com/ehabkost/qemu.git tags/python-next-pull-request
> >> >
> >> > for you to fetch changes up to d24e417866f85229de1b75bc5c0a1d942451a842:
> >> >
> >> >   configure: Require Python >= 3.5 (2019-10-25 16:34:57 -0300)
> >> >
> >> > ----------------------------------------------------------------
> >> > Require Python >= 3.5 to build QEMU
> >> >
> >> > ----------------------------------------------------------------
> >>
> >> I can't apply this until we've fixed the tests/vm netbsd setup to
> >> not use Python 2.
> >
> > Fixing tests/vm/netbsd is being tricky.  It looks like the
> > configure patch will have to wait until after QEMU 4.2.0.  :(
> 
> I've posted fixes for the netbsd serial install but there are still
> problems with the tests including what looks like a fairly serious
> failure in the async code.

This sounds like a known "feature": QEMU expects clients to be
constantly reading from chardev sockets until the socket is
closed.  Otherwise, VCPU threads may block waiting for the socket
to be writeable.


> 
> >
> >>
> >> Have you tried a test run with Travis/etc/etc to check that none of
> >> those CI configs need updating to have python3 available ?
> >
> > I have tested this pull request on Shippable, and I will take a
> > look at Travis.  I'd appreciate help from the CI system
> > maintainers (CCed) for the rest, as I don't have accounts in all
> > our CI systems.
> 
> Setting up accounts on the others doesn't take long. I use the
> CustomCIStatus template to instantiate all the buttons for my various
> maintainer branches on the wiki, e.g.:
> 
>   {{CustomCIStatus|user=stsquad|repo=qemu|branch=testing/next|ship_proj=5885eac43b653a0f00fa97f5}}
> 
> which means I just have to glance at the button state rather than going
> through each individual CI's status pages.

This is awesome.  Thanks for the tip!

> 
> > Do we expect maintainers to test their pull requests in all CI
> > systems listed at the QEMU wiki[1]?  Do we have an official list
> > of CI systems that you consider to be pull request blockers?
> 
> Well they all catch various things but none of them catch all the things
> Peter's PR processing does. Historically Travis has been allowed to
> slide because of test instability and timeouts. Having said that last I
> checked everything was green so breaking any of the main CIs
> (Travis/Shippable/Cirrus/Gitlab) indicates there is a problem that needs
> to be fixed.

Manually checking if 5 different CI systems are green wouldn't be
reasonable to me, but the CustomCIStatus template will be useful.
Daniel P. Berrangé Nov. 6, 2019, 10:36 a.m. UTC | #6
On Tue, Nov 05, 2019 at 08:25:03PM +0000, Alex Bennée wrote:
> 
> Eduardo Habkost <ehabkost@redhat.com> writes:
> 
> > On Thu, Oct 31, 2019 at 08:12:01AM +0000, Peter Maydell wrote:
> >> On Fri, 25 Oct 2019 at 21:34, Eduardo Habkost <ehabkost@redhat.com> wrote:
> >> >
> >> > The following changes since commit 03bf012e523ecdf047ac56b2057950247256064d:
> >> >
> >> >   Merge remote-tracking branch 'remotes/kevin/tags/for-upstream' into staging (2019-10-25 14:59:53 +0100)
> >> >
> >> > are available in the Git repository at:
> >> >
> >> >   git://github.com/ehabkost/qemu.git tags/python-next-pull-request
> >> >
> >> > for you to fetch changes up to d24e417866f85229de1b75bc5c0a1d942451a842:
> >> >
> >> >   configure: Require Python >= 3.5 (2019-10-25 16:34:57 -0300)
> >> >
> >> > ----------------------------------------------------------------
> >> > Require Python >= 3.5 to build QEMU
> >> >
> >> > ----------------------------------------------------------------
> >>
> >> I can't apply this until we've fixed the tests/vm netbsd setup to
> >> not use Python 2.
> >
> > Fixing tests/vm/netbsd is being tricky.  It looks like the
> > configure patch will have to wait until after QEMU 4.2.0.  :(
> 
> I've posted fixes for the netbsd serial install but there are still
> problems with the tests including what looks like a fairly serious
> failure in the async code.

At what point do we declare that NetBSD CI is broken and is no longer
considered a supported platform from POV of blocking the merging of
PULL requests ? It has been preventing the dropping of python2 for
quite a while now. It isn't the end of the world in this particular
case, as dropping py2 is mostly just a cleanup, but I feel like we
might benefit from setting expectations for ongoing platform maintenance,
otherwise these kind of issues could drag on indefinitely.

Regards,
Daniel
Peter Maydell Nov. 6, 2019, 11:17 a.m. UTC | #7
On Wed, 6 Nov 2019 at 10:36, Daniel P. Berrangé <berrange@redhat.com> wrote:
> At what point do we declare that NetBSD CI is broken and is no longer
> considered a supported platform from POV of blocking the merging of
> PULL requests ? It has been preventing the dropping of python2 for
> quite a while now. It isn't the end of the world in this particular
> case, as dropping py2 is mostly just a cleanup, but I feel like we
> might benefit from setting expectations for ongoing platform maintenance,
> otherwise these kind of issues could drag on indefinitely.

It works fine for me, and it means we have coverage of a host
OS we otherwise would not. To me that is definitely more important
than being able to drop Python 2 support. Also, AIUI the problem
that's blocking updating the NetBSD image isn't related to
NetBSD at all but is a bug in some combination of QEMU itself
and our test framework -- both of those are things we need to
fix anyway.

thanks
-- PMM
Alex Bennée Nov. 6, 2019, 11:48 a.m. UTC | #8
Peter Maydell <peter.maydell@linaro.org> writes:

> On Wed, 6 Nov 2019 at 10:36, Daniel P. Berrangé <berrange@redhat.com> wrote:
>> At what point do we declare that NetBSD CI is broken and is no longer
>> considered a supported platform from POV of blocking the merging of
>> PULL requests ? It has been preventing the dropping of python2 for
>> quite a while now. It isn't the end of the world in this particular
>> case, as dropping py2 is mostly just a cleanup, but I feel like we
>> might benefit from setting expectations for ongoing platform maintenance,
>> otherwise these kind of issues could drag on indefinitely.
>
> It works fine for me, and it means we have coverage of a host
> OS we otherwise would not. To me that is definitely more important
> than being able to drop Python 2 support. Also, AIUI the problem
> that's blocking updating the NetBSD image isn't related to
> NetBSD at all but is a bug in some combination of QEMU itself
> and our test framework

These have been addressed in:

  Subject: [PATCH  v1 0/6] testing/next (netbsd stuff)
  Date: Mon,  4 Nov 2019 17:36:48 +0000
  Message-Id: <20191104173654.30125-1-alex.bennee@linaro.org>

but I have a non-trivial failure rate running tests (~20% of runs fail)

> -- both of those are things we need to
> fix anyway.
>
> thanks
> -- PMM


--
Alex Bennée