mbox

[PULL,v8,00/86] Misc QEMU patches for 2020-09-24

Message ID 20200930195850.278104-1-pbonzini@redhat.com
State New
Headers show

Pull-request

https://gitlab.com/bonzini/qemu.git tags/for-upstream

Message

Paolo Bonzini Sept. 30, 2020, 7:58 p.m. UTC
The following changes since commit cbba3dc6ea3fc9aa66e9f9eb41051536e3ad7cd0:

  Merge remote-tracking branch 'remotes/mst/tags/for_upstream' into staging (2020-09-30 11:40:38 +0100)

are available in the Git repository at:

  https://gitlab.com/bonzini/qemu.git tags/for-upstream

for you to fetch changes up to 37aeb7a28ddbf52dd25dd53ae1b8391bc2287858:

  hw/net/can: Correct Kconfig dependencies (2020-09-30 19:11:37 +0200)

----------------------------------------------------------------
* SCSI fix (Dmitry, Li Feng, Li Qiang)
* memory API fixes (Eduardo)
* removal of deprecated '-numa node', 'cpu-add', '-smp' (Igor)
* ACPI fix for VMBus (Jon)
* relocatable install (myself)
* always remove docker containers (myself)
* serial cleanups (Philippe)
* vmware cpuid leaf for tsc and apic frequency (Sunil)
* KVM_FEATURE_ASYNC_PF_INT support (Vitaly)
* i386 XSAVE bugfix (Xiaoyao)
* QOM developer documentation in docs/devel (Eduardo)
* new checkpatch tests (Dov)
* x86_64 syscall fix (Douglas)
* interrupt-based APF fix (Vitaly)
* always create kvmclock (Vitaly)
* fix bios-tables-test (Eduardo)
* KVM PV features cleanup (myself)
* CAN FD (Pavel)

meson:
* fixes (Marc-André, Max, Stefan, Alexander, myself)
* moved libmpathpersist, cocoa, malloc tests (myself)
* support for 0.56 introspected test dependencies (myself)

----------------------------------------------------------------
Alexander Bulekov (1):
      oss-fuzz: move linker arg to fix coverage-build

Anthony PERARD (1):
      meson: fix installation of keymaps

Claudio Fontana (1):
      tests: add missing genh dependency

Daniel P. Berrangé (1):
      char: fix logging when chardev write fails

Dmitry Fomichev (1):
      scsi-generic: Fix HM-zoned device scan

Douglas Crosher (1):
      helper_syscall x86_64: clear exception_is_int

Dov Murik (1):
      checkpatch: Detect '%#' or '%0#' in printf-style format strings

Eduardo Habkost (10):
      memory: Convert IOMMUMemoryRegionClass doc comment to kernel-doc
      qom: Document all function parameters in doc comments
      qom: Use kernel-doc private/public tags in structs
      qom: Use ``code`` Sphinx syntax where appropriate
      qom: Add kernel-doc markup to introduction doc comment
      qom: Reformat section titles using Sphinx syntax
      qom: Indent existing code examples
      qom: Add code block markup to all code blocks
      docs: Create docs/devel/qom.rst
      bios-tables-test: Remove kernel-irqchip=off option

Igor Mammedov (4):
      numa: drop support for '-numa node' (without memory specified)
      doc: Cleanup "'-mem-path' fallback to RAM" deprecation text
      numa: remove fixup numa_state->num_nodes to MAX_NODES
      smp: drop support for deprecated (invalid topologies)

Jan Charvat (5):
      net/can: Initial host SocketCan support for CAN FD.
      hw/net/can: sja1000 ignore CAN FD frames
      net/can: Add can_dlc2len and can_len2dlc for CAN FD.
      hw/net/can/ctucafd: Add CTU CAN FD core register definitions.
      hw/net/can: CTU CAN FD IP open hardware core emulation.

Jon Doron (1):
      acpi: i386: Move VMBus DSDT entry to SB

Li Feng (1):
      vhost-scsi: support inflight io track

Li Qiang (2):
      hw: megasas: return -1 when 'megasas_map_sgl' fails
      hw: megasas: consider 'iov_count=0' is an error in megasas_map_sgl

Marc-André Lureau (2):
      meson: fix MSI rule
      meson: error out if qemu_suffix starts with /

Paolo Bonzini (28):
      meson: clean up build_by_default
      ninjatool: rebuild multi-output targets if outputs are missing
      meson: move libudev test
      meson: move libmpathpersist test
      meson: extend libmpathpersist test for static linking
      configure: move malloc_trim/tcmalloc/jemalloc to meson
      configure: fix --meson=/path/to/meson
      configure: move cocoa option to Meson
      configure: do not limit Hypervisor.framework test to Darwin
      meson: qtest: set "depends" correctly
      mtest2make: add support for introspected test dependencies
      meson: report accelerator support
      oslib: do not call g_strdup from qemu_get_exec_dir
      fuzz: use qemu_get_exec_dir
      oslib-posix: default exec_dir to bindir
      cutils: introduce get_relocated_path
      oslib-posix: relocate path to /var
      module: relocate path to modules
      net: relocate paths to helpers and scripts
      vl: relocate paths to data directories
      vl: relocate path to configuration file
      qemu-bridge-helper: relocate path to default ACL
      qga: relocate path to default configuration and hook
      ui: relocate paths to icons and translations
      configure: use a platform-neutral prefix
      tests/tcg: reinstate or replace desired parts of rules.mak
      docs: Move object.h overview doc comment to qom.rst
      target/i386: kvm: do not use kvm_check_extension to find paravirtual capabilities

Pavel Pisa (2):
      hw/net/can: Documentation for CTU CAN FD IP open hardware core emulation.
      hw/net/can: Correct Kconfig dependencies

Philippe Mathieu-Daudé (16):
      configure: Do not intent to build WHPX on 32-bit host
      hw/mips/mipssim: Use MMIO serial device on fake ISA I/O
      hw/char/serial: Remove TYPE_SERIAL_IO
      hw/char/serial: Assert serial_ioport_read/write offset fits 8 bytes
      hw/char/serial: Replace commented DPRINTF() by trace event
      hw/char/serial: Remove old DEBUG_SERIAL commented code
      hw/char/serial: Rename I/O read/write trace events
      hw/char/serial: Make 'wakeup' property boolean
      hw/char/serial-{isa, pci}: Alias QDEV properties from generic serial object
      hw/i386/q35: Remove unreachable Xen code on Q35 machine
      exec: Remove MemoryRegion::global_locking field
      hw/i386/xen: Rename X86/PC specific function as xen_hvm_init_pc()
      sysemu/xen: Add missing 'exec/cpu-common.h' header for ram_addr_t type
      stubs: Split accelerator / hardware related stubs
      hw/xen: Split x86-specific declaration from generic hardware ones
      typedefs: Restrict PCMachineState to 'hw/i386/pc.h'

Stefan Hajnoczi (2):
      MAINTAINERS: add Paolo Bonzini as RCU maintainer
      configure: rename QEMU_GA_MSI_ENABLED to CONFIG_QGA_MSI

Stefano Garzarella (1):
      checkpatch: avoid error on cover letter files

Sunil Muthuswamy (1):
      WHPX: vmware cpuid leaf for tsc and apic frequency

Vitaly Kuznetsov (3):
      target/i386: support KVM_FEATURE_ASYNC_PF_INT
      target/i386: Fix VM migration when interrupt based APF is enabled
      target/i386: always create kvmclock device

Xiaoyao Li (1):
      i386/cpu: Clear FEAT_XSAVE_COMP_{LO,HI} when XSAVE is not available

 MAINTAINERS                           |  18 +
 Makefile                              |   4 +-
 accel/stubs/xen-stub.c                |  41 +-
 accel/tcg/cputlb.c                    |   4 +-
 chardev/char.c                        |  13 +
 configure                             | 237 ++-------
 docs/can.txt                          | 113 +++-
 docs/devel/build-system.rst           |   5 +
 docs/devel/index.rst                  |   1 +
 docs/devel/qom.rst                    | 378 +++++++++++++
 docs/system/deprecated.rst            |  57 +-
 exec.c                                |   2 +-
 hw/char/serial-isa.c                  |   4 +-
 hw/char/serial-pci.c                  |   3 +-
 hw/char/serial.c                      |  65 +--
 hw/char/trace-events                  |   5 +-
 hw/core/machine.c                     |   1 -
 hw/core/numa.c                        |  59 ---
 hw/i386/acpi-build.c                  |  12 +-
 hw/i386/kvm/clock.c                   |   7 +-
 hw/i386/microvm.c                     |   2 +-
 hw/i386/pc.c                          |   1 +
 hw/i386/pc_piix.c                     |  16 +-
 hw/i386/pc_q35.c                      |  19 +-
 hw/i386/xen/xen-hvm.c                 |   3 +-
 hw/mips/mipssim.c                     |   5 +-
 hw/net/Kconfig                        |  16 +-
 hw/net/can/can_sja1000.c              |  31 +-
 hw/net/can/ctu_can_fd_frame.h         | 189 +++++++
 hw/net/can/ctu_can_fd_regs.h          | 971 ++++++++++++++++++++++++++++++++++
 hw/net/can/ctucan_core.c              | 696 ++++++++++++++++++++++++
 hw/net/can/ctucan_core.h              | 127 +++++
 hw/net/can/ctucan_pci.c               | 281 ++++++++++
 hw/net/can/meson.build                |   2 +
 hw/ppc/spapr.c                        |   1 -
 hw/scsi/megasas.c                     |   6 +-
 hw/scsi/scsi-generic.c                |  10 +-
 hw/scsi/vhost-scsi-common.c           |  27 +
 include/exec/memory.h                 |  64 ++-
 include/hw/boards.h                   |   2 -
 include/hw/char/serial.h              |  11 +-
 include/hw/i386/pc.h                  |   7 +-
 include/hw/kvm/clock.h                |   4 +-
 include/hw/virtio/vhost-scsi-common.h |   2 +
 include/hw/xen/xen-x86.h              |  15 +
 include/hw/xen/xen.h                  |   2 -
 include/net/can_emu.h                 |  12 +-
 include/net/net.h                     |   4 +-
 include/qemu-common.h                 |   1 -
 include/qemu/cutils.h                 |  12 +
 include/qemu/osdep.h                  |   8 +-
 include/qemu/typedefs.h               |   1 -
 include/qom/object.h                  | 547 ++++---------------
 include/scsi/constants.h              |   1 +
 include/sysemu/numa.h                 |   4 -
 include/sysemu/sysemu.h               |   2 +-
 include/sysemu/xen.h                  |   2 +
 meson.build                           | 202 +++++--
 meson_options.txt                     |   9 +
 net/can/can_core.c                    |  36 ++
 net/can/can_socketcan.c               |  47 +-
 net/tap.c                             |  26 +-
 os-posix.c                            |  24 -
 os-win32.c                            |  11 -
 pc-bios/keymaps/meson.build           |   2 +
 qemu-bridge-helper.c                  |   9 +-
 qga/main.c                            |   8 +-
 qga/meson.build                       |   1 +
 scripts/checkpatch.pl                 |  12 +-
 scripts/mtest2make.py                 |  19 +-
 scripts/ninjatool.py                  |   3 +
 softmmu/memory.c                      |   6 -
 softmmu/vl.c                          |  43 +-
 stubs/meson.build                     |   1 +
 stubs/xen-hw-stub.c                   |  50 ++
 target/i386/cpu.c                     |   5 +-
 target/i386/cpu.h                     |   1 +
 target/i386/kvm.c                     |  45 +-
 target/i386/kvm_i386.h                |   1 +
 target/i386/machine.c                 |  19 +
 target/i386/seg_helper.c              |   1 +
 target/i386/whp-dispatch.h            |   3 +-
 target/i386/whpx-all.c                |  96 +++-
 tests/fp/meson.build                  |   2 -
 tests/meson.build                     |   9 +-
 tests/qemu-iotests/meson.build        |   3 +-
 tests/qtest/bios-tables-test.c        |   3 +-
 tests/qtest/fuzz/fuzz.c               |  12 +-
 tests/qtest/meson.build               |   8 +-
 tests/tcg/Makefile.qemu               |  13 +-
 tests/tcg/configure.sh                |   4 +-
 trace/meson.build                     |   1 -
 ui/gtk.c                              |  10 +-
 ui/meson.build                        |   2 +-
 ui/sdl2.c                             |   9 +-
 ui/shader/meson.build                 |   1 -
 util/cutils.c                         |  61 +++
 util/module.c                         |  10 +-
 util/oslib-posix.c                    |  35 +-
 util/oslib-win32.c                    |  14 +-
 100 files changed, 3842 insertions(+), 1168 deletions(-)
 create mode 100644 docs/devel/qom.rst
 create mode 100644 hw/net/can/ctu_can_fd_frame.h
 create mode 100644 hw/net/can/ctu_can_fd_regs.h
 create mode 100644 hw/net/can/ctucan_core.c
 create mode 100644 hw/net/can/ctucan_core.h
 create mode 100644 hw/net/can/ctucan_pci.c
 create mode 100644 include/hw/xen/xen-x86.h
 create mode 100644 stubs/xen-hw-stub.c

Comments

Peter Maydell Oct. 1, 2020, 2:28 p.m. UTC | #1
On Wed, 30 Sep 2020 at 21:04, Paolo Bonzini <pbonzini@redhat.com> wrote:
>
> The following changes since commit cbba3dc6ea3fc9aa66e9f9eb41051536e3ad7cd0:
>
>   Merge remote-tracking branch 'remotes/mst/tags/for_upstream' into staging (2020-09-30 11:40:38 +0100)
>
> are available in the Git repository at:
>
>   https://gitlab.com/bonzini/qemu.git tags/for-upstream
>
> for you to fetch changes up to 37aeb7a28ddbf52dd25dd53ae1b8391bc2287858:
>
>   hw/net/can: Correct Kconfig dependencies (2020-09-30 19:11:37 +0200)
>
> ----------------------------------------------------------------
> * SCSI fix (Dmitry, Li Feng, Li Qiang)
> * memory API fixes (Eduardo)
> * removal of deprecated '-numa node', 'cpu-add', '-smp' (Igor)
> * ACPI fix for VMBus (Jon)
> * relocatable install (myself)
> * always remove docker containers (myself)
> * serial cleanups (Philippe)
> * vmware cpuid leaf for tsc and apic frequency (Sunil)
> * KVM_FEATURE_ASYNC_PF_INT support (Vitaly)
> * i386 XSAVE bugfix (Xiaoyao)
> * QOM developer documentation in docs/devel (Eduardo)
> * new checkpatch tests (Dov)
> * x86_64 syscall fix (Douglas)
> * interrupt-based APF fix (Vitaly)
> * always create kvmclock (Vitaly)
> * fix bios-tables-test (Eduardo)
> * KVM PV features cleanup (myself)
> * CAN FD (Pavel)
>
> meson:
> * fixes (Marc-André, Max, Stefan, Alexander, myself)
> * moved libmpathpersist, cocoa, malloc tests (myself)
> * support for 0.56 introspected test dependencies (myself)
>


Applied, thanks.

Please update the changelog at https://wiki.qemu.org/ChangeLog/5.2
for any user-visible changes.

-- PMM
Yonggang Luo Oct. 1, 2020, 3:41 p.m. UTC | #2
On Thu, Oct 1, 2020 at 10:36 PM Peter Maydell <peter.maydell@linaro.org>
wrote:
>
> On Wed, 30 Sep 2020 at 21:04, Paolo Bonzini <pbonzini@redhat.com> wrote:
> >
> > The following changes since commit
cbba3dc6ea3fc9aa66e9f9eb41051536e3ad7cd0:
> >
> >   Merge remote-tracking branch 'remotes/mst/tags/for_upstream' into
staging (2020-09-30 11:40:38 +0100)
> >
> > are available in the Git repository at:
> >
> >   https://gitlab.com/bonzini/qemu.git tags/for-upstream
> >
> > for you to fetch changes up to 37aeb7a28ddbf52dd25dd53ae1b8391bc2287858:
> >
> >   hw/net/can: Correct Kconfig dependencies (2020-09-30 19:11:37 +0200)
> >
> > ----------------------------------------------------------------
> > * SCSI fix (Dmitry, Li Feng, Li Qiang)
> > * memory API fixes (Eduardo)
> > * removal of deprecated '-numa node', 'cpu-add', '-smp' (Igor)
> > * ACPI fix for VMBus (Jon)
> > * relocatable install (myself)
> > * always remove docker containers (myself)
> > * serial cleanups (Philippe)
> > * vmware cpuid leaf for tsc and apic frequency (Sunil)
> > * KVM_FEATURE_ASYNC_PF_INT support (Vitaly)
> > * i386 XSAVE bugfix (Xiaoyao)
> > * QOM developer documentation in docs/devel (Eduardo)
> > * new checkpatch tests (Dov)
> > * x86_64 syscall fix (Douglas)
> > * interrupt-based APF fix (Vitaly)
> > * always create kvmclock (Vitaly)
> > * fix bios-tables-test (Eduardo)
> > * KVM PV features cleanup (myself)
> > * CAN FD (Pavel)
> >
> > meson:
> > * fixes (Marc-André, Max, Stefan, Alexander, myself)
> > * moved libmpathpersist, cocoa, malloc tests (myself)
> > * support for 0.56 introspected test dependencies (myself)
> >
>
>
> Applied, thanks.
>
> Please update the changelog at https://wiki.qemu.org/ChangeLog/5.2
> for any user-visible changes.
>
> -- PMM
>
Long awating, wonderfull

--
         此致
礼
罗勇刚
Yours
    sincerely,
Yonggang Luo
Michal Prívozník Oct. 2, 2020, 3:58 p.m. UTC | #3
On 9/30/20 9:58 PM, Paolo Bonzini wrote:
 >
> Eduardo Habkost (10):
<snip/>
>        docs: Create docs/devel/qom.rst

cd442a45db60a1a72fcf980c24bd1227f13f8a87 is the first bad commit

Sorry for noticing this earlier, but is this known? The build starts 
failing for me after this commit:

/usr/bin/sphinx-build -Dversion=5.1.50 -Drelease= -W 
-Ddepfile=docs/devel.d -Ddepfile_stamp=docs/devel.stamp -b html -d 
/home/zippy/work/qemu/qemu.git/build/docs/devel.p 
/home/zippy/work/qemu/qemu.git/docs/devel 
/home/zippy/work/qemu/qemu.git/build/docs/devel
Running Sphinx v3.2.1
building [mo]: targets for 0 po files that are out of date
building [html]: targets for 20 source files that are out of date
updating environment: [new config] 20 added, 0 changed, 0 removed
reading sources... [100%] testing 
 
 


Warning, treated as error:
/home/zippy/work/qemu/qemu.git/docs/../include/qom/object.h:747:Error in 
declarator
If declarator-id with parameters (e.g., 'void f(int arg)'):
   Invalid C declaration: Expected identifier in nested name. [error at 24]
     object_initialize_child ( parent,  propname,  child,  type)
     ------------------------^
If parenthesis in noptr-declarator (e.g., 'void (*f(int arg))(double)'):
   Error in declarator or parameters
   Invalid C declaration: Expecting "(" in parameters. [error at 32]
     object_initialize_child ( parent,  propname,  child,  type)
     --------------------------------^

make[1]: *** [Makefile.ninja:9898: docs/devel.stamp] Error 2
make[1]: *** Deleting file 'docs/devel.stamp'
make[1]: Leaving directory '/home/zippy/work/qemu/qemu.git/build'
make: *** [GNUmakefile:11: all] Error 2


Michal
Eduardo Habkost Oct. 2, 2020, 4:22 p.m. UTC | #4
On Fri, Oct 02, 2020 at 05:58:55PM +0200, Michal Prívozník wrote:
> On 9/30/20 9:58 PM, Paolo Bonzini wrote:
> >
> > Eduardo Habkost (10):
> <snip/>
> >        docs: Create docs/devel/qom.rst
> 
> cd442a45db60a1a72fcf980c24bd1227f13f8a87 is the first bad commit
> 
> Sorry for noticing this earlier, but is this known? The build starts failing
> for me after this commit:
> 
> /usr/bin/sphinx-build -Dversion=5.1.50 -Drelease= -W -Ddepfile=docs/devel.d
> -Ddepfile_stamp=docs/devel.stamp -b html -d
> /home/zippy/work/qemu/qemu.git/build/docs/devel.p
> /home/zippy/work/qemu/qemu.git/docs/devel
> /home/zippy/work/qemu/qemu.git/build/docs/devel
> Running Sphinx v3.2.1
> building [mo]: targets for 0 po files that are out of date
> building [html]: targets for 20 source files that are out of date
> updating environment: [new config] 20 added, 0 changed, 0 removed
> reading sources... [100%] testing
> 
> 
> 
> 
> Warning, treated as error:
> /home/zippy/work/qemu/qemu.git/docs/../include/qom/object.h:747:Error in
> declarator
> If declarator-id with parameters (e.g., 'void f(int arg)'):
>   Invalid C declaration: Expected identifier in nested name. [error at 24]
>     object_initialize_child ( parent,  propname,  child,  type)
>     ------------------------^
> If parenthesis in noptr-declarator (e.g., 'void (*f(int arg))(double)'):
>   Error in declarator or parameters
>   Invalid C declaration: Expecting "(" in parameters. [error at 32]
>     object_initialize_child ( parent,  propname,  child,  type)
>     --------------------------------^
> 
> make[1]: *** [Makefile.ninja:9898: docs/devel.stamp] Error 2
> make[1]: *** Deleting file 'docs/devel.stamp'
> make[1]: Leaving directory '/home/zippy/work/qemu/qemu.git/build'
> make: *** [GNUmakefile:11: all] Error 2

I can't reproduce it using Sphinx v2.2.2.  I'm still trying to
understand what exactly the error means.

I really wish we used virtualenv + requirements.txt to require a
specific version of Sphinx instead of wasting time dealing a wide
range of Sphinx versions.
Paolo Bonzini Oct. 2, 2020, 4:27 p.m. UTC | #5
On 02/10/20 17:58, Michal Prívozník wrote:
>>
> 
> cd442a45db60a1a72fcf980c24bd1227f13f8a87 is the first bad commit
> 
> Sorry for noticing this earlier, but is this known? The build starts
> failing for me after this commit:
> 
> /usr/bin/sphinx-build -Dversion=5.1.50 -Drelease= -W
> -Ddepfile=docs/devel.d -Ddepfile_stamp=docs/devel.stamp -b html -d
> /home/zippy/work/qemu/qemu.git/build/docs/devel.p
> /home/zippy/work/qemu/qemu.git/docs/devel
> /home/zippy/work/qemu/qemu.git/build/docs/devel
> Running Sphinx v3.2.1
> building [mo]: targets for 0 po files that are out of date
> building [html]: targets for 20 source files that are out of date
> updating environment: [new config] 20 added, 0 changed, 0 removed
> reading sources... [100%] testing

No, this is new.  It works with older versions of Sphinx (I have 2.2.2
despite being on Fedora 32 which is pretty recent).

For now Sphinx 3 is not supported by kerneldoc, we probably should apply
a patch like

https://www.spinics.net/lists/linux-doc/msg83277.html

Paolo
Eduardo Habkost Oct. 2, 2020, 5:01 p.m. UTC | #6
On Fri, Oct 02, 2020 at 06:27:35PM +0200, Paolo Bonzini wrote:
> On 02/10/20 17:58, Michal Prívozník wrote:
> >>
> > 
> > cd442a45db60a1a72fcf980c24bd1227f13f8a87 is the first bad commit
> > 
> > Sorry for noticing this earlier, but is this known? The build starts
> > failing for me after this commit:
> > 
> > /usr/bin/sphinx-build -Dversion=5.1.50 -Drelease= -W
> > -Ddepfile=docs/devel.d -Ddepfile_stamp=docs/devel.stamp -b html -d
> > /home/zippy/work/qemu/qemu.git/build/docs/devel.p
> > /home/zippy/work/qemu/qemu.git/docs/devel
> > /home/zippy/work/qemu/qemu.git/build/docs/devel
> > Running Sphinx v3.2.1
> > building [mo]: targets for 0 po files that are out of date
> > building [html]: targets for 20 source files that are out of date
> > updating environment: [new config] 20 added, 0 changed, 0 removed
> > reading sources... [100%] testing
> 
> No, this is new.  It works with older versions of Sphinx (I have 2.2.2
> despite being on Fedora 32 which is pretty recent).
> 
> For now Sphinx 3 is not supported by kerneldoc, we probably should apply
> a patch like
> 
> https://www.spinics.net/lists/linux-doc/msg83277.html

Weird, because we don't use the cdomain extension, and the patch
above only disables the cdomain extension.

In either case, supporting only Sphinx 2.x by now sounds
reasonable.  virtualenv+pip makes it simple to install and run a
specific Sphinx version, for people who really need it.
Michal Prívozník Oct. 2, 2020, 5:26 p.m. UTC | #7
On 10/2/20 6:22 PM, Eduardo Habkost wrote:
> On Fri, Oct 02, 2020 at 05:58:55PM +0200, Michal Prívozník wrote:
>> On 9/30/20 9:58 PM, Paolo Bonzini wrote:
>>>
>>> Eduardo Habkost (10):
>> <snip/>
>>>         docs: Create docs/devel/qom.rst
>>
>> cd442a45db60a1a72fcf980c24bd1227f13f8a87 is the first bad commit
>>
>> Sorry for noticing this earlier, but is this known? The build starts failing
>> for me after this commit:
>>
>> /usr/bin/sphinx-build -Dversion=5.1.50 -Drelease= -W -Ddepfile=docs/devel.d
>> -Ddepfile_stamp=docs/devel.stamp -b html -d
>> /home/zippy/work/qemu/qemu.git/build/docs/devel.p
>> /home/zippy/work/qemu/qemu.git/docs/devel
>> /home/zippy/work/qemu/qemu.git/build/docs/devel
>> Running Sphinx v3.2.1
>> building [mo]: targets for 0 po files that are out of date
>> building [html]: targets for 20 source files that are out of date
>> updating environment: [new config] 20 added, 0 changed, 0 removed
>> reading sources... [100%] testing
>>
>>
>>
>>
>> Warning, treated as error:
>> /home/zippy/work/qemu/qemu.git/docs/../include/qom/object.h:747:Error in
>> declarator
>> If declarator-id with parameters (e.g., 'void f(int arg)'):
>>    Invalid C declaration: Expected identifier in nested name. [error at 24]
>>      object_initialize_child ( parent,  propname,  child,  type)
>>      ------------------------^
>> If parenthesis in noptr-declarator (e.g., 'void (*f(int arg))(double)'):
>>    Error in declarator or parameters
>>    Invalid C declaration: Expecting "(" in parameters. [error at 32]
>>      object_initialize_child ( parent,  propname,  child,  type)
>>      --------------------------------^
>>
>> make[1]: *** [Makefile.ninja:9898: docs/devel.stamp] Error 2
>> make[1]: *** Deleting file 'docs/devel.stamp'
>> make[1]: Leaving directory '/home/zippy/work/qemu/qemu.git/build'
>> make: *** [GNUmakefile:11: all] Error 2
> 
> I can't reproduce it using Sphinx v2.2.2.  I'm still trying to
> understand what exactly the error means.
> 

Same here.

> I really wish we used virtualenv + requirements.txt to require a
> specific version of Sphinx instead of wasting time dealing a wide
> range of Sphinx versions.
> 

I already have a patch that I keep locally to build with v3:

diff --git a/docs/qemu-option-trace.rst.inc b/docs/qemu-option-trace.rst.inc
index 7e09773a9c..ae83f6a1a8 100644
--- a/docs/qemu-option-trace.rst.inc
+++ b/docs/qemu-option-trace.rst.inc
@@ -1,7 +1,7 @@

  Specify tracing options.

-.. option:: [enable=]PATTERN
+.. option:: enable=PATTERN

    Immediately enable events matching *PATTERN*
    (either event name or a globbing pattern).  This option is only


That said, I'm not objecting to requiring v2 for now and switching to v3 
later.


But interestingly, through trial and error I've came across this hack, 
which allows me to build again. I have no idea why it works:

diff --git i/include/qom/object.h w/include/qom/object.h
index 27aaa67e63..59c729ebb7 100644
--- i/include/qom/object.h
+++ w/include/qom/object.h
@@ -762,13 +762,14 @@ bool object_initialize_child_with_propsv(Object 
*parentobj,
   *                                      child, sizeof(*child), type,
   *                                      &error_abort, NULL)
   */
-#define object_initialize_child(parent, propname, child, type)          \
-    object_initialize_child_internal((parent), (propname),              \
-                                     (child), sizeof(*(child)), (type))
  void object_initialize_child_internal(Object *parent, const char 
*propname,
                                        void *child, size_t size,
                                        const char *type);

+#define object_initialize_child(parent, propname, child, type)          \
+    object_initialize_child_internal((parent), (propname),              \
+                                     (child), sizeof(*(child)), (type))
+
  /**
   * object_dynamic_cast:
   * @obj: The object to cast.
Paolo Bonzini Oct. 2, 2020, 5:30 p.m. UTC | #8
On 02/10/20 19:26, Michal Prívozník wrote:
> On 10/2/20 6:22 PM, Eduardo Habkost wrote:
>> On Fri, Oct 02, 2020 at 05:58:55PM +0200, Michal Prívozník wrote:
>>> On 9/30/20 9:58 PM, Paolo Bonzini wrote:
>>>>
>>>> Eduardo Habkost (10):
>>> <snip/>
>>>>         docs: Create docs/devel/qom.rst
>>>
>>> cd442a45db60a1a72fcf980c24bd1227f13f8a87 is the first bad commit
>>>
>>> Sorry for noticing this earlier, but is this known? The build starts
>>> failing
>>> for me after this commit:
>>>
>>> /usr/bin/sphinx-build -Dversion=5.1.50 -Drelease= -W
>>> -Ddepfile=docs/devel.d
>>> -Ddepfile_stamp=docs/devel.stamp -b html -d
>>> /home/zippy/work/qemu/qemu.git/build/docs/devel.p
>>> /home/zippy/work/qemu/qemu.git/docs/devel
>>> /home/zippy/work/qemu/qemu.git/build/docs/devel
>>> Running Sphinx v3.2.1
>>> building [mo]: targets for 0 po files that are out of date
>>> building [html]: targets for 20 source files that are out of date
>>> updating environment: [new config] 20 added, 0 changed, 0 removed
>>> reading sources... [100%] testing
>>>
>>>
>>>
>>>
>>> Warning, treated as error:
>>> /home/zippy/work/qemu/qemu.git/docs/../include/qom/object.h:747:Error in
>>> declarator
>>> If declarator-id with parameters (e.g., 'void f(int arg)'):
>>>    Invalid C declaration: Expected identifier in nested name. [error
>>> at 24]
>>>      object_initialize_child ( parent,  propname,  child,  type)
>>>      ------------------------^
>>> If parenthesis in noptr-declarator (e.g., 'void (*f(int arg))(double)'):
>>>    Error in declarator or parameters
>>>    Invalid C declaration: Expecting "(" in parameters. [error at 32]
>>>      object_initialize_child ( parent,  propname,  child,  type)
>>>      --------------------------------^
>>>
>>> make[1]: *** [Makefile.ninja:9898: docs/devel.stamp] Error 2
>>> make[1]: *** Deleting file 'docs/devel.stamp'
>>> make[1]: Leaving directory '/home/zippy/work/qemu/qemu.git/build'
>>> make: *** [GNUmakefile:11: all] Error 2
>>
>> I can't reproduce it using Sphinx v2.2.2.  I'm still trying to
>> understand what exactly the error means.
>>
> 
> Same here.
> 
>> I really wish we used virtualenv + requirements.txt to require a
>> specific version of Sphinx instead of wasting time dealing a wide
>> range of Sphinx versions.
>>
> 
> I already have a patch that I keep locally to build with v3:
> 
> diff --git a/docs/qemu-option-trace.rst.inc
> b/docs/qemu-option-trace.rst.inc
> index 7e09773a9c..ae83f6a1a8 100644
> --- a/docs/qemu-option-trace.rst.inc
> +++ b/docs/qemu-option-trace.rst.inc
> @@ -1,7 +1,7 @@
> 
>  Specify tracing options.
> 
> -.. option:: [enable=]PATTERN
> +.. option:: enable=PATTERN
> 
>    Immediately enable events matching *PATTERN*
>    (either event name or a globbing pattern).  This option is only
> 
> 
> That said, I'm not objecting to requiring v2 for now and switching to v3
> later.
> 
> 
> But interestingly, through trial and error I've came across this hack,
> which allows me to build again. I have no idea why it works:
> 
> diff --git i/include/qom/object.h w/include/qom/object.h
> index 27aaa67e63..59c729ebb7 100644
> --- i/include/qom/object.h
> +++ w/include/qom/object.h
> @@ -762,13 +762,14 @@ bool object_initialize_child_with_propsv(Object
> *parentobj,
>   *                                      child, sizeof(*child), type,
>   *                                      &error_abort, NULL)
>   */
> -#define object_initialize_child(parent, propname, child, type)          \
> -    object_initialize_child_internal((parent), (propname),              \
> -                                     (child), sizeof(*(child)), (type))
>  void object_initialize_child_internal(Object *parent, const char
> *propname,
>                                        void *child, size_t size,
>                                        const char *type);
> 
> +#define object_initialize_child(parent, propname, child, type)          \
> +    object_initialize_child_internal((parent), (propname),              \
> +                                     (child), sizeof(*(child)), (type))
> +

The error is due to kerneldoc treating the macro definition like a
function, so that makes sense.  If the docs look good (no reference to
object_initialize_child_internal) then the patch can be applied.

Paolo
Eduardo Habkost Oct. 2, 2020, 5:55 p.m. UTC | #9
On Fri, Oct 02, 2020 at 07:30:17PM +0200, Paolo Bonzini wrote:
> On 02/10/20 19:26, Michal Prívozník wrote:
> > On 10/2/20 6:22 PM, Eduardo Habkost wrote:
> >> On Fri, Oct 02, 2020 at 05:58:55PM +0200, Michal Prívozník wrote:
> >>> On 9/30/20 9:58 PM, Paolo Bonzini wrote:
> >>>>
> >>>> Eduardo Habkost (10):
> >>> <snip/>
> >>>>         docs: Create docs/devel/qom.rst
> >>>
> >>> cd442a45db60a1a72fcf980c24bd1227f13f8a87 is the first bad commit
> >>>
> >>> Sorry for noticing this earlier, but is this known? The build starts
> >>> failing
> >>> for me after this commit:
> >>>
> >>> /usr/bin/sphinx-build -Dversion=5.1.50 -Drelease= -W
> >>> -Ddepfile=docs/devel.d
> >>> -Ddepfile_stamp=docs/devel.stamp -b html -d
> >>> /home/zippy/work/qemu/qemu.git/build/docs/devel.p
> >>> /home/zippy/work/qemu/qemu.git/docs/devel
> >>> /home/zippy/work/qemu/qemu.git/build/docs/devel
> >>> Running Sphinx v3.2.1
> >>> building [mo]: targets for 0 po files that are out of date
> >>> building [html]: targets for 20 source files that are out of date
> >>> updating environment: [new config] 20 added, 0 changed, 0 removed
> >>> reading sources... [100%] testing
> >>>
> >>>
> >>>
> >>>
> >>> Warning, treated as error:
> >>> /home/zippy/work/qemu/qemu.git/docs/../include/qom/object.h:747:Error in
> >>> declarator
> >>> If declarator-id with parameters (e.g., 'void f(int arg)'):
> >>>    Invalid C declaration: Expected identifier in nested name. [error
> >>> at 24]
> >>>      object_initialize_child ( parent,  propname,  child,  type)
> >>>      ------------------------^
> >>> If parenthesis in noptr-declarator (e.g., 'void (*f(int arg))(double)'):
> >>>    Error in declarator or parameters
> >>>    Invalid C declaration: Expecting "(" in parameters. [error at 32]
> >>>      object_initialize_child ( parent,  propname,  child,  type)
> >>>      --------------------------------^
> >>>
> >>> make[1]: *** [Makefile.ninja:9898: docs/devel.stamp] Error 2
> >>> make[1]: *** Deleting file 'docs/devel.stamp'
> >>> make[1]: Leaving directory '/home/zippy/work/qemu/qemu.git/build'
> >>> make: *** [GNUmakefile:11: all] Error 2
> >>
> >> I can't reproduce it using Sphinx v2.2.2.  I'm still trying to
> >> understand what exactly the error means.
> >>
> > 
> > Same here.
> > 
> >> I really wish we used virtualenv + requirements.txt to require a
> >> specific version of Sphinx instead of wasting time dealing a wide
> >> range of Sphinx versions.
> >>
> > 
> > I already have a patch that I keep locally to build with v3:
> > 
> > diff --git a/docs/qemu-option-trace.rst.inc
> > b/docs/qemu-option-trace.rst.inc
> > index 7e09773a9c..ae83f6a1a8 100644
> > --- a/docs/qemu-option-trace.rst.inc
> > +++ b/docs/qemu-option-trace.rst.inc
> > @@ -1,7 +1,7 @@
> > 
> >  Specify tracing options.
> > 
> > -.. option:: [enable=]PATTERN
> > +.. option:: enable=PATTERN
> > 
> >    Immediately enable events matching *PATTERN*
> >    (either event name or a globbing pattern).  This option is only
> > 
> > 
> > That said, I'm not objecting to requiring v2 for now and switching to v3
> > later.
> > 
> > 
> > But interestingly, through trial and error I've came across this hack,
> > which allows me to build again. I have no idea why it works:
> > 
> > diff --git i/include/qom/object.h w/include/qom/object.h
> > index 27aaa67e63..59c729ebb7 100644
> > --- i/include/qom/object.h
> > +++ w/include/qom/object.h
> > @@ -762,13 +762,14 @@ bool object_initialize_child_with_propsv(Object
> > *parentobj,
> >   *                                      child, sizeof(*child), type,
> >   *                                      &error_abort, NULL)
> >   */
> > -#define object_initialize_child(parent, propname, child, type)          \
> > -    object_initialize_child_internal((parent), (propname),              \
> > -                                     (child), sizeof(*(child)), (type))
> >  void object_initialize_child_internal(Object *parent, const char
> > *propname,
> >                                        void *child, size_t size,
> >                                        const char *type);
> > 
> > +#define object_initialize_child(parent, propname, child, type)          \
> > +    object_initialize_child_internal((parent), (propname),              \
> > +                                     (child), sizeof(*(child)), (type))
> > +
> 
> The error is due to kerneldoc treating the macro definition like a
> function, so that makes sense.  If the docs look good (no reference to
> object_initialize_child_internal) then the patch can be applied.

The patch makes the document show object_initialize_child_internal().
Eduardo Habkost Oct. 2, 2020, 6:24 p.m. UTC | #10
On Fri, Oct 02, 2020 at 06:27:35PM +0200, Paolo Bonzini wrote:
> On 02/10/20 17:58, Michal Prívozník wrote:
> >>
> > 
> > cd442a45db60a1a72fcf980c24bd1227f13f8a87 is the first bad commit
> > 
> > Sorry for noticing this earlier, but is this known? The build starts
> > failing for me after this commit:
> > 
> > /usr/bin/sphinx-build -Dversion=5.1.50 -Drelease= -W
> > -Ddepfile=docs/devel.d -Ddepfile_stamp=docs/devel.stamp -b html -d
> > /home/zippy/work/qemu/qemu.git/build/docs/devel.p
> > /home/zippy/work/qemu/qemu.git/docs/devel
> > /home/zippy/work/qemu/qemu.git/build/docs/devel
> > Running Sphinx v3.2.1
> > building [mo]: targets for 0 po files that are out of date
> > building [html]: targets for 20 source files that are out of date
> > updating environment: [new config] 20 added, 0 changed, 0 removed
> > reading sources... [100%] testing
> 
> No, this is new.  It works with older versions of Sphinx (I have 2.2.2
> despite being on Fedora 32 which is pretty recent).
> 
> For now Sphinx 3 is not supported by kerneldoc, we probably should apply
> a patch like
> 
> https://www.spinics.net/lists/linux-doc/msg83277.html

We already have Sphinx 3.x hacks inside our fork of kernel-doc,
see commit 152d1967f650 ("kernel-doc: Use c:struct for Sphinx 3.0
and later").

If we want to keep deviating from upstream kernel-doc, the
following patch seems to work.  Do we want to?

Signed-off-by: Eduardo Habkost <ehabkost@redhat.com>
---
diff --git a/scripts/kernel-doc b/scripts/kernel-doc
index 40ad782e342..03b49380426 100755
--- a/scripts/kernel-doc
+++ b/scripts/kernel-doc
@@ -838,6 +838,13 @@ sub output_function_rst(%) {
 	$lineprefix = "";
 	output_highlight_rst($args{'purpose'});
 	$start = "\n\n**Syntax**\n\n  ``";
+    } elsif ($args{'functiontype'} eq "") {
+	# this is a macro, Sphinx 3.x requires c:macro::
+	if ((split(/\./, $sphinx_version))[0] >= 3) {
+		print ".. c:macro:: ";
+	} else {
+		print ".. c:function:: ";
+	}
     } else {
 	print ".. c:function:: ";
     }
> 
> Paolo
>
Paolo Bonzini Oct. 2, 2020, 6:45 p.m. UTC | #11
On 02/10/20 20:24, Eduardo Habkost wrote:
> On Fri, Oct 02, 2020 at 06:27:35PM +0200, Paolo Bonzini wrote:
>> On 02/10/20 17:58, Michal Prívozník wrote:
>>>>
>>>
>>> cd442a45db60a1a72fcf980c24bd1227f13f8a87 is the first bad commit
>>>
>>> Sorry for noticing this earlier, but is this known? The build starts
>>> failing for me after this commit:
>>>
>>> /usr/bin/sphinx-build -Dversion=5.1.50 -Drelease= -W
>>> -Ddepfile=docs/devel.d -Ddepfile_stamp=docs/devel.stamp -b html -d
>>> /home/zippy/work/qemu/qemu.git/build/docs/devel.p
>>> /home/zippy/work/qemu/qemu.git/docs/devel
>>> /home/zippy/work/qemu/qemu.git/build/docs/devel
>>> Running Sphinx v3.2.1
>>> building [mo]: targets for 0 po files that are out of date
>>> building [html]: targets for 20 source files that are out of date
>>> updating environment: [new config] 20 added, 0 changed, 0 removed
>>> reading sources... [100%] testing
>>
>> No, this is new.  It works with older versions of Sphinx (I have 2.2.2
>> despite being on Fedora 32 which is pretty recent).
>>
>> For now Sphinx 3 is not supported by kerneldoc, we probably should apply
>> a patch like
>>
>> https://www.spinics.net/lists/linux-doc/msg83277.html
> 
> We already have Sphinx 3.x hacks inside our fork of kernel-doc,
> see commit 152d1967f650 ("kernel-doc: Use c:struct for Sphinx 3.0
> and later").
> 
> If we want to keep deviating from upstream kernel-doc, the
> following patch seems to work.  Do we want to?
> 
> Signed-off-by: Eduardo Habkost <ehabkost@redhat.com>
> ---
> diff --git a/scripts/kernel-doc b/scripts/kernel-doc
> index 40ad782e342..03b49380426 100755
> --- a/scripts/kernel-doc
> +++ b/scripts/kernel-doc
> @@ -838,6 +838,13 @@ sub output_function_rst(%) {
>  	$lineprefix = "";
>  	output_highlight_rst($args{'purpose'});
>  	$start = "\n\n**Syntax**\n\n  ``";
> +    } elsif ($args{'functiontype'} eq "") {
> +	# this is a macro, Sphinx 3.x requires c:macro::
> +	if ((split(/\./, $sphinx_version))[0] >= 3) {
> +		print ".. c:macro:: ";
> +	} else {
> +		print ".. c:function:: ";
> +	}
>      } else {
>  	print ".. c:function:: ";
>      }
>>
>> Paolo
>>
> 

I can try sending these to upstream Linux, too.

Paolo
Peter Maydell Oct. 2, 2020, 6:52 p.m. UTC | #12
On Fri, 2 Oct 2020 at 19:45, Paolo Bonzini <pbonzini@redhat.com> wrote:
> On 02/10/20 20:24, Eduardo Habkost wrote:
> > We already have Sphinx 3.x hacks inside our fork of kernel-doc,
> > see commit 152d1967f650 ("kernel-doc: Use c:struct for Sphinx 3.0
> > and later").
> >
> > If we want to keep deviating from upstream kernel-doc, the
> > following patch seems to work.  Do we want to?

> I can try sending these to upstream Linux, too.

I did mention the c:struct change we're carrying to Jon Corbet
when I sent him the missing-close-paren fix to kernel-doc
earlier this year:
http://lkml.iu.edu/hypermail/linux/kernel/2004.1/08760.html
I think it probably mostly needs somebody to try these changes
in the context of producing the kernel docs and see if they
are sufficient to fix kernel doc production too...

thanks
-- PMM
Michal Prívozník Oct. 7, 2020, 10:57 a.m. UTC | #13
On 10/2/20 8:24 PM, Eduardo Habkost wrote:
> On Fri, Oct 02, 2020 at 06:27:35PM +0200, Paolo Bonzini wrote:
>> On 02/10/20 17:58, Michal Prívozník wrote:
>>>>
>>>
>>> cd442a45db60a1a72fcf980c24bd1227f13f8a87 is the first bad commit
>>>
>>> Sorry for noticing this earlier, but is this known? The build starts
>>> failing for me after this commit:
>>>
>>> /usr/bin/sphinx-build -Dversion=5.1.50 -Drelease= -W
>>> -Ddepfile=docs/devel.d -Ddepfile_stamp=docs/devel.stamp -b html -d
>>> /home/zippy/work/qemu/qemu.git/build/docs/devel.p
>>> /home/zippy/work/qemu/qemu.git/docs/devel
>>> /home/zippy/work/qemu/qemu.git/build/docs/devel
>>> Running Sphinx v3.2.1
>>> building [mo]: targets for 0 po files that are out of date
>>> building [html]: targets for 20 source files that are out of date
>>> updating environment: [new config] 20 added, 0 changed, 0 removed
>>> reading sources... [100%] testing
>>
>> No, this is new.  It works with older versions of Sphinx (I have 2.2.2
>> despite being on Fedora 32 which is pretty recent).
>>
>> For now Sphinx 3 is not supported by kerneldoc, we probably should apply
>> a patch like
>>
>> https://www.spinics.net/lists/linux-doc/msg83277.html
> 
> We already have Sphinx 3.x hacks inside our fork of kernel-doc,
> see commit 152d1967f650 ("kernel-doc: Use c:struct for Sphinx 3.0
> and later").
> 
> If we want to keep deviating from upstream kernel-doc, the
> following patch seems to work.  Do we want to?
> 
> Signed-off-by: Eduardo Habkost <ehabkost@redhat.com>
> ---
> diff --git a/scripts/kernel-doc b/scripts/kernel-doc
> index 40ad782e342..03b49380426 100755
> --- a/scripts/kernel-doc
> +++ b/scripts/kernel-doc
> @@ -838,6 +838,13 @@ sub output_function_rst(%) {
>   	$lineprefix = "";
>   	output_highlight_rst($args{'purpose'});
>   	$start = "\n\n**Syntax**\n\n  ``";
> +    } elsif ($args{'functiontype'} eq "") {
> +	# this is a macro, Sphinx 3.x requires c:macro::
> +	if ((split(/\./, $sphinx_version))[0] >= 3) {
> +		print ".. c:macro:: ";
> +	} else {
> +		print ".. c:function:: ";
> +	}
>       } else {
>   	print ".. c:function:: ";
>       }
>>
>> Paolo
>>
> 

I can confirm that this fixes the build for me.

Michal