From patchwork Wed May 17 17:44:13 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Paolo Bonzini X-Patchwork-Id: 1782826 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=nongnu.org (client-ip=209.51.188.17; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=) Authentication-Results: legolas.ozlabs.org; dkim=pass (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=a5GhvQNX; dkim-atps=neutral Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-ECDSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4QM0s9379Mz20dn for ; Thu, 18 May 2023 03:46:21 +1000 (AEST) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1pzLDV-0006sx-Oz; Wed, 17 May 2023 13:45:38 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pzLDP-0006jw-Vm for qemu-devel@nongnu.org; Wed, 17 May 2023 13:45:35 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.129.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pzLDM-0003yp-QS for qemu-devel@nongnu.org; Wed, 17 May 2023 13:45:30 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1684345527; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=GITj+Z9gCB+Vro48aLKzM2p8bxrlRde0LfE7vxRAVLM=; b=a5GhvQNX8j7WmyCNWskjDism5/AxkastbpoGRy1wCY9ztOOQnHrC0WRWrYNLBDLDhkcrlv BWDHKnyAovDnXmWEkrzzlQ4EmL/oOCF8MYLl+ltAR8EAJ1RidJn8pnDKcrRpzSKmA4g7X1 Ng3G1lAlobfaHCbWtGNL7weV80Wk+Qc= Received: from mail-wm1-f69.google.com (mail-wm1-f69.google.com [209.85.128.69]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-63-ebneM5wtNPymTykwwgmb_A-1; Wed, 17 May 2023 13:45:26 -0400 X-MC-Unique: ebneM5wtNPymTykwwgmb_A-1 Received: by mail-wm1-f69.google.com with SMTP id 5b1f17b1804b1-3f429072212so4935355e9.1 for ; Wed, 17 May 2023 10:45:26 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1684345524; x=1686937524; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=GITj+Z9gCB+Vro48aLKzM2p8bxrlRde0LfE7vxRAVLM=; b=SJAzLYu3kH5VjwyN2m4MZdVLnJuT499DHwgWTWGJ2+Pt/XrNDgsUPEmkfGSI2wiKla oUQJ/rxDNPwzBEH5LeEt+0x//OVGNVYh7TUbnWAoVWtr4G9Dbktjj+UJ16fYrMJ2BP+6 TFc4NjQgG+GnsglYVZPFCEpz87JL1KhqlKI0gJijxy+aHP494QINxBUXmm0/kIu+Zxai cjoyne3oDXtkIB/VA7/vjFR5kEx5yiol0E0BePDTwMTwcNe6PaqrW8NCqMyyIbKFF8EU wx8lAcLcI/YaEbInNYkgWpCeq1c6zPi1BCM4Y/0DFY3XdZRGwRk3zXqQZ4FLtMLaTc1W NcfA== X-Gm-Message-State: AC+VfDyqUg1Tqgm0hdt2GQy2Gp4ask0SqH/1QFxTkpOwBQzMXOWax2U2 EiniqsVqPQOTb05/s9CRJzjvSWPVjabYBxebtEjLpscOtVQBLY3XKGuwbETW3tJxkx0+pmvr2cy UDNqSSVLWuJw7fxJKRNl5eDYYSKkFzrQFGuFa9D2zQ9MlI0snw2fdBBBSzeJdm+PdkMTQ8xJDO2 A= X-Received: by 2002:a7b:cd11:0:b0:3f4:b6bc:bd93 with SMTP id f17-20020a7bcd11000000b003f4b6bcbd93mr18643366wmj.35.1684345524743; Wed, 17 May 2023 10:45:24 -0700 (PDT) X-Google-Smtp-Source: ACHHUZ6ioTLsRi0ZFm9Ruhg5D1NbpHKq2QcAt+R2LWtSR7bivIdBfW3H4LEjKhlOS76t+21HSO3gzw== X-Received: by 2002:a7b:cd11:0:b0:3f4:b6bc:bd93 with SMTP id f17-20020a7bcd11000000b003f4b6bcbd93mr18643353wmj.35.1684345524347; Wed, 17 May 2023 10:45:24 -0700 (PDT) Received: from [192.168.10.118] ([2001:b07:6468:f312:9af8:e5f5:7516:fa89]) by smtp.gmail.com with ESMTPSA id m39-20020a05600c3b2700b003f07ef4e3e0sm5339482wms.0.2023.05.17.10.45.23 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 17 May 2023 10:45:23 -0700 (PDT) From: Paolo Bonzini To: qemu-devel@nongnu.org Cc: Emanuele Giuseppe Esposito Subject: [PULL 01/68] target/i386: add support for FLUSH_L1D feature Date: Wed, 17 May 2023 19:44:13 +0200 Message-Id: <20230517174520.887405-2-pbonzini@redhat.com> X-Mailer: git-send-email 2.40.1 In-Reply-To: <20230517174520.887405-1-pbonzini@redhat.com> References: <20230517174520.887405-1-pbonzini@redhat.com> MIME-Version: 1.0 Received-SPF: pass client-ip=170.10.129.124; envelope-from=pbonzini@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org From: Emanuele Giuseppe Esposito As reported by Intel's doc: "L1D_FLUSH: Writeback and invalidate the L1 data cache" If this cpu feature is present in host, allow QEMU to choose whether to show it to the guest too. One disadvantage of not exposing it is that the guest will report a non existing vulnerability in /sys/devices/system/cpu/vulnerabilities/mmio_stale_data because the mitigation is present only when the cpu has (FLUSH_L1D and MD_CLEAR) or FB_CLEAR features enabled. Signed-off-by: Emanuele Giuseppe Esposito Message-Id: <20230201135759.555607-2-eesposit@redhat.com> Signed-off-by: Paolo Bonzini --- target/i386/cpu.c | 2 +- target/i386/cpu.h | 2 ++ 2 files changed, 3 insertions(+), 1 deletion(-) diff --git a/target/i386/cpu.c b/target/i386/cpu.c index 4187759f106f..e5c813199a0d 100644 --- a/target/i386/cpu.c +++ b/target/i386/cpu.c @@ -863,7 +863,7 @@ FeatureWordInfo feature_word_info[FEATURE_WORDS] = { "tsx-ldtrk", NULL, NULL /* pconfig */, "arch-lbr", NULL, NULL, "amx-bf16", "avx512-fp16", "amx-tile", "amx-int8", "spec-ctrl", "stibp", - NULL, "arch-capabilities", "core-capability", "ssbd", + "flush-l1d", "arch-capabilities", "core-capability", "ssbd", }, .cpuid = { .eax = 7, diff --git a/target/i386/cpu.h b/target/i386/cpu.h index 8ade71ab555d..3c6f9226e1f1 100644 --- a/target/i386/cpu.h +++ b/target/i386/cpu.h @@ -899,6 +899,8 @@ uint64_t x86_cpu_get_supported_feature_word(FeatureWord w, #define CPUID_7_0_EDX_SPEC_CTRL (1U << 26) /* Single Thread Indirect Branch Predictors */ #define CPUID_7_0_EDX_STIBP (1U << 27) +/* Flush L1D cache */ +#define CPUID_7_0_EDX_FLUSH_L1D (1U << 28) /* Arch Capabilities */ #define CPUID_7_0_EDX_ARCH_CAPABILITIES (1U << 29) /* Core Capability */ From patchwork Wed May 17 17:44:14 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Paolo Bonzini X-Patchwork-Id: 1782838 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=nongnu.org (client-ip=209.51.188.17; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=) Authentication-Results: legolas.ozlabs.org; dkim=pass (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=SxNhIRGz; dkim-atps=neutral Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-ECDSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4QM0vZ2Whmz20dq for ; Thu, 18 May 2023 03:48:26 +1000 (AEST) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1pzLDW-0006uy-Ny; Wed, 17 May 2023 13:45:38 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pzLDS-0006mA-4x for qemu-devel@nongnu.org; Wed, 17 May 2023 13:45:35 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pzLDO-0003zI-Gr for qemu-devel@nongnu.org; Wed, 17 May 2023 13:45:33 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1684345529; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=9EoFhTMSuj6MdzeaASZZUUxMEPiDwMGTBjRSt3wNO30=; b=SxNhIRGz1dNFGU/tHrdEVLOWJQQDZE+UlalJ9FVTrJdon5Z1/xqybcHgvjrdPI3bcF+sIQ lSTbt57ZGblMmGdVBi0cn5MwGjlXJKtY/qRIIjSAxw/GUwFkqBeRnHjDGbSpzBtcIsM7t8 +xUA8AiyXLxtUpvT+ZGznDyQqfhK5h0= Received: from mail-wr1-f70.google.com (mail-wr1-f70.google.com [209.85.221.70]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-248-YY8ITvs3OGWt7W59Rm2PtQ-1; Wed, 17 May 2023 13:45:27 -0400 X-MC-Unique: YY8ITvs3OGWt7W59Rm2PtQ-1 Received: by mail-wr1-f70.google.com with SMTP id ffacd0b85a97d-30634323dfeso490120f8f.3 for ; Wed, 17 May 2023 10:45:27 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1684345526; x=1686937526; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=9EoFhTMSuj6MdzeaASZZUUxMEPiDwMGTBjRSt3wNO30=; b=aYniz6eTL2ExXyO18ZwcV88SWL7jCZ7DYFEXfThmU2YI8SkcsGPSOKMYPv23LohkYm Ns8YL2gkhcnZAMr6KKCg8mO7C0zjCqEKza2ZNen3FiCv+ZNaWMRB0xVjKo/5jpA4Oj6a iJMeKyDd4tH5Do2lCwaJDuEcB6aXFnSUm9kzmG0Edtvx6VzOrQjyysxu9PYzwKC8Wkfg aFu2ouVL9bVz44PKhV94j1pAQAzIoSPnbbHCzhVjAHA347tZojv7HGrbeuH92+kiYPwh JKOryBMkUUL7HzJ0G1U7s+pAcrsigAJ2Xa2f3sju/jleAzY9bMQ+Wmyndq7L2jvdsAzE 3d7g== X-Gm-Message-State: AC+VfDzUsz37JfHrL4K5aJ5wVjGbQ2AktJMiywGxPg4xjJxgQwOcVIKs PqYqy0+b8RkO0oHry64GTdCrBXoLHWMEuJsG7xCAKej9CQGv28ApOAoaSweE8nccg1J6NX0tO7V XRLNtfgGIZS0nubstr+9cCh9054ZgZGavDJYZ2Nh9X/Af+UoWYDSi6YjRNolyEQPFZoWg1+i2Bb 0= X-Received: by 2002:adf:f3c1:0:b0:307:95d1:d7d0 with SMTP id g1-20020adff3c1000000b0030795d1d7d0mr1330625wrp.39.1684345526115; Wed, 17 May 2023 10:45:26 -0700 (PDT) X-Google-Smtp-Source: ACHHUZ5Ty4qSOgIWhhFB9GodF8Fnhb/hMbOpik0zchP0N6X2gCM0R1YeXIs5IN5ROgInBGkqy5CndQ== X-Received: by 2002:adf:f3c1:0:b0:307:95d1:d7d0 with SMTP id g1-20020adff3c1000000b0030795d1d7d0mr1330612wrp.39.1684345525816; Wed, 17 May 2023 10:45:25 -0700 (PDT) Received: from [192.168.10.118] ([2001:b07:6468:f312:9af8:e5f5:7516:fa89]) by smtp.gmail.com with ESMTPSA id g18-20020a5d5552000000b003047dc162f7sm3237610wrw.67.2023.05.17.10.45.24 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 17 May 2023 10:45:25 -0700 (PDT) From: Paolo Bonzini To: qemu-devel@nongnu.org Cc: Emanuele Giuseppe Esposito Subject: [PULL 02/68] target/i386: add support for FB_CLEAR feature Date: Wed, 17 May 2023 19:44:14 +0200 Message-Id: <20230517174520.887405-3-pbonzini@redhat.com> X-Mailer: git-send-email 2.40.1 In-Reply-To: <20230517174520.887405-1-pbonzini@redhat.com> References: <20230517174520.887405-1-pbonzini@redhat.com> MIME-Version: 1.0 Received-SPF: pass client-ip=170.10.133.124; envelope-from=pbonzini@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org From: Emanuele Giuseppe Esposito As reported by the Intel's doc: "FB_CLEAR: The processor will overwrite fill buffer values as part of MD_CLEAR operations with the VERW instruction. On these processors, L1D_FLUSH does not overwrite fill buffer values." If this cpu feature is present in host, allow QEMU to choose whether to show it to the guest too. One disadvantage of not exposing it is that the guest will report a non existing vulnerability in /sys/devices/system/cpu/vulnerabilities/mmio_stale_data because the mitigation is present only when the cpu has (FLUSH_L1D and MD_CLEAR) or FB_CLEAR features enabled. Signed-off-by: Emanuele Giuseppe Esposito Message-Id: <20230201135759.555607-3-eesposit@redhat.com> Signed-off-by: Paolo Bonzini --- target/i386/cpu.c | 2 +- target/i386/cpu.h | 1 + 2 files changed, 2 insertions(+), 1 deletion(-) diff --git a/target/i386/cpu.c b/target/i386/cpu.c index e5c813199a0d..a61cd6d99d1f 100644 --- a/target/i386/cpu.c +++ b/target/i386/cpu.c @@ -1050,7 +1050,7 @@ FeatureWordInfo feature_word_info[FEATURE_WORDS] = { "ssb-no", "mds-no", "pschange-mc-no", "tsx-ctrl", "taa-no", NULL, NULL, NULL, NULL, NULL, NULL, NULL, - NULL, NULL, NULL, NULL, + NULL, "fb-clear", NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, diff --git a/target/i386/cpu.h b/target/i386/cpu.h index 3c6f9226e1f1..7201a71de863 100644 --- a/target/i386/cpu.h +++ b/target/i386/cpu.h @@ -1018,6 +1018,7 @@ uint64_t x86_cpu_get_supported_feature_word(FeatureWord w, #define MSR_ARCH_CAP_PSCHANGE_MC_NO (1U << 6) #define MSR_ARCH_CAP_TSX_CTRL_MSR (1U << 7) #define MSR_ARCH_CAP_TAA_NO (1U << 8) +#define MSR_ARCH_CAP_FB_CLEAR (1U << 17) #define MSR_CORE_CAP_SPLIT_LOCK_DETECT (1U << 5) From patchwork Wed May 17 17:44:15 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Paolo Bonzini X-Patchwork-Id: 1782835 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=nongnu.org (client-ip=209.51.188.17; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=) Authentication-Results: legolas.ozlabs.org; dkim=pass (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=DYg9Fv7B; dkim-atps=neutral Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-ECDSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4QM0tL55yrz20dn for ; Thu, 18 May 2023 03:47:22 +1000 (AEST) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1pzLDY-0006y2-EB; Wed, 17 May 2023 13:45:40 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pzLDS-0006mU-Qb for qemu-devel@nongnu.org; Wed, 17 May 2023 13:45:35 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pzLDQ-0003zd-GT for qemu-devel@nongnu.org; Wed, 17 May 2023 13:45:34 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1684345531; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=7MVkYSUnOf9teWZaYW9AhoFuts4lEDIpBaOEEfStXpQ=; b=DYg9Fv7Bzp8nQr3wwbP8ilQUsxP++S9i+W/qtOXL65CfXNSFgTQ7Or4WxozSYAeP9nGQtq GgjM8LwZlW6e+m870oM/pVTGQjua2bdtvsdXdOJ2XnEJo9mwf/UwV+A6z5IC1imN9Kt4Yd EpdsnwpaWE1TiWD479KFwYhO5qcj1Ko= Received: from mail-wm1-f70.google.com (mail-wm1-f70.google.com [209.85.128.70]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-260-FrpbDyiTMDuftoLdOhDZvQ-1; Wed, 17 May 2023 13:45:29 -0400 X-MC-Unique: FrpbDyiTMDuftoLdOhDZvQ-1 Received: by mail-wm1-f70.google.com with SMTP id 5b1f17b1804b1-3f518f2900eso7556255e9.2 for ; Wed, 17 May 2023 10:45:29 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1684345528; x=1686937528; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=7MVkYSUnOf9teWZaYW9AhoFuts4lEDIpBaOEEfStXpQ=; b=OhMTQYRb0XbErM/bvQ1nwZcIvN6Hg9qWtHAtE3iAUHuSxPuWnFVfOQ9/ykj8+yPnNV AhbMor9PhFMiT/DNGiAH34Z5RWo88+8VnmrlV2PNud6fNxyWvGRzL0RncrCwVlsu+uBk fxOu5q3LSZzbTYRUzcFt/5TUupAcWRmVwod/A6GgdJMKSVSMqUEMVpYjPH9NdZdEwEx/ 6Tx7OKWjr01TyfaRW9ky3yFMNjxUqAxdFTN2d3AxlEnCyBWMDSvwlwqY452wZHECr99O yH/vu9LMJ63Y1lk+x5dqd5agTLYKY/CnEzcErLt6KtUvEkiP95omnps1rzI53yjy32bX 6CPw== X-Gm-Message-State: AC+VfDxfrFXfqpT2PuIbTpV2DMZOBD1L7ZvD9r6w3nF06OZE38n+Tltr Ao/hAMnRozUPTlAFHFg25HFo/1CHYWnb7q7tSGyab80saDqw4b7n8iqULGv6xI8LjgLVMyBfthm taIxCIFonSHjhQCFGiF36OQ4BA1O5SMWB2tfBayfPt8ypNXBoAL+lwn16WU23Oc5Y7/3fyKv3tq o= X-Received: by 2002:a05:600c:2292:b0:3f4:fa56:3186 with SMTP id 18-20020a05600c229200b003f4fa563186mr11219602wmf.14.1684345527903; Wed, 17 May 2023 10:45:27 -0700 (PDT) X-Google-Smtp-Source: ACHHUZ665wpEFu3tJO3ViabSJmCUrIOIU83WMF4eLyPNgi/7vIDToQ/EE/ZtGzRgbnUDWPZYzhXrsQ== X-Received: by 2002:a05:600c:2292:b0:3f4:fa56:3186 with SMTP id 18-20020a05600c229200b003f4fa563186mr11219584wmf.14.1684345527476; Wed, 17 May 2023 10:45:27 -0700 (PDT) Received: from [192.168.10.118] ([2001:b07:6468:f312:9af8:e5f5:7516:fa89]) by smtp.gmail.com with ESMTPSA id p4-20020a05600c358400b003f1738d0d13sm3960075wmq.1.2023.05.17.10.45.26 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 17 May 2023 10:45:27 -0700 (PDT) From: Paolo Bonzini To: qemu-devel@nongnu.org Cc: Gabriele Svelto , qemu-stable@nongnu.org Subject: [PULL 03/68] target/i386: fix operand size for VCOMI/VUCOMI instructions Date: Wed, 17 May 2023 19:44:15 +0200 Message-Id: <20230517174520.887405-4-pbonzini@redhat.com> X-Mailer: git-send-email 2.40.1 In-Reply-To: <20230517174520.887405-1-pbonzini@redhat.com> References: <20230517174520.887405-1-pbonzini@redhat.com> MIME-Version: 1.0 Received-SPF: pass client-ip=170.10.133.124; envelope-from=pbonzini@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=unavailable autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Compared to other SSE instructions, VUCOMISx and VCOMISx are different: the single and double precision versions are distinguished through a prefix, however they use no-prefix and 0x66 for SS and SD respectively. Scalar values usually are associated with 0xF2 and 0xF3. Because of these, they incorrectly perform a 128-bit memory load instead of a 32- or 64-bit load. Fix this by writing a custom decoding function. I tested that the reproducer is fixed and the test-avx output does not change. Reported-by: Gabriele Svelto Resolves: https://gitlab.com/qemu-project/qemu/-/issues/1637 Fixes: f8d19eec0d53 ("target/i386: reimplement 0x0f 0x28-0x2f, add AVX", 2022-10-18) Cc: qemu-stable@nongnu.org Signed-off-by: Paolo Bonzini --- target/i386/tcg/decode-new.c.inc | 15 +++++++++++++-- 1 file changed, 13 insertions(+), 2 deletions(-) diff --git a/target/i386/tcg/decode-new.c.inc b/target/i386/tcg/decode-new.c.inc index 4fdd87750bea..48fefaffdf63 100644 --- a/target/i386/tcg/decode-new.c.inc +++ b/target/i386/tcg/decode-new.c.inc @@ -783,6 +783,17 @@ static void decode_0F2D(DisasContext *s, CPUX86State *env, X86OpEntry *entry, ui *entry = *decode_by_prefix(s, opcodes_0F2D); } +static void decode_VxCOMISx(DisasContext *s, CPUX86State *env, X86OpEntry *entry, uint8_t *b) +{ + /* + * VUCOMISx and VCOMISx are different and use no-prefix and 0x66 for SS and SD + * respectively. Scalar values usually are associated with 0xF2 and 0xF3, for + * which X86_VEX_REPScalar exists, but here it has to be decoded by hand. + */ + entry->s1 = entry->s2 = (s->prefix & PREFIX_DATA ? X86_SIZE_sd : X86_SIZE_ss); + entry->gen = (*b == 0x2E ? gen_VUCOMI : gen_VCOMI); +} + static void decode_sse_unary(DisasContext *s, CPUX86State *env, X86OpEntry *entry, uint8_t *b) { if (!(s->prefix & (PREFIX_REPZ | PREFIX_REPNZ))) { @@ -871,8 +882,8 @@ static const X86OpEntry opcodes_0F[256] = { [0x2B] = X86_OP_GROUP0(0F2B), [0x2C] = X86_OP_GROUP0(0F2C), [0x2D] = X86_OP_GROUP0(0F2D), - [0x2E] = X86_OP_ENTRY3(VUCOMI, None,None, V,x, W,x, vex4 p_00_66), - [0x2F] = X86_OP_ENTRY3(VCOMI, None,None, V,x, W,x, vex4 p_00_66), + [0x2E] = X86_OP_GROUP3(VxCOMISx, None,None, V,x, W,x, vex3 p_00_66), /* VUCOMISS/SD */ + [0x2F] = X86_OP_GROUP3(VxCOMISx, None,None, V,x, W,x, vex3 p_00_66), /* VCOMISS/SD */ [0x38] = X86_OP_GROUP0(0F38), [0x3a] = X86_OP_GROUP0(0F3A), From patchwork Wed May 17 17:44:16 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Paolo Bonzini X-Patchwork-Id: 1782830 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=nongnu.org (client-ip=209.51.188.17; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=) Authentication-Results: legolas.ozlabs.org; dkim=pass (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=Z5oLmvPQ; dkim-atps=neutral Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-ECDSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4QM0sf0R8Lz20dq for ; Thu, 18 May 2023 03:46:46 +1000 (AEST) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1pzLDj-00075s-Ny; Wed, 17 May 2023 13:45:51 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pzLDZ-00070D-NX for qemu-devel@nongnu.org; Wed, 17 May 2023 13:45:41 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.129.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pzLDY-00041q-85 for qemu-devel@nongnu.org; Wed, 17 May 2023 13:45:41 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1684345539; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=MYIgHKkXiwxtH3/oSlQBUrXOjLZvBMixiKEGCXviiBE=; b=Z5oLmvPQvTFWJPfuyIdE+K2ldszBTdAWEmZ339RsqhWEOYBF+vegWZeIyIAar8ltpadKBj JKAhBfqxzuBLQFo/gNPMgPxMiR7xXFGYHyuib+m4wsmJ7+WZ96HVL1rm12O0yNByliOzf7 kWmMUKsyEA7bPI3RGUaEaln/2khQDNA= Received: from mail-wm1-f72.google.com (mail-wm1-f72.google.com [209.85.128.72]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-665-RW43ElxfOIyMSPSVlpktGQ-1; Wed, 17 May 2023 13:45:31 -0400 X-MC-Unique: RW43ElxfOIyMSPSVlpktGQ-1 Received: by mail-wm1-f72.google.com with SMTP id 5b1f17b1804b1-3f4fa463282so7683745e9.0 for ; Wed, 17 May 2023 10:45:31 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1684345530; x=1686937530; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=MYIgHKkXiwxtH3/oSlQBUrXOjLZvBMixiKEGCXviiBE=; b=Gtp/Fqh+jb9FnitS6g+N8fzxiA94QSiDUJYFr66dGgYa9mB8Gnjxa7Kgq8LW20qUK/ CoWLl+3sJaJDr0SoiEg4uv+2aqGxOkiZS7oxX1sbiiOV7jzU6SEE9DLSzNWLtGTXrp13 Fa2jrdR/JuY1F3AS36EHoXFeNIV1TIVBK0E7G9UyGPWEpiKszKXqnR4CKroBMt3gJdzL 2pq1Xy3wu+gUjkolnUtC77pQEvLW8e3nLN+kfprIUPoqWOene1GNPBlyfAerTAMBQsnZ FoZxnE1jOzKb+l2ZBx+rwvCzknB/3jhQXiRWBC+Zu3/MZtAi9V7SJTo1JOoSeGA+jqLk x7Pw== X-Gm-Message-State: AC+VfDzHwG14p2P6H1Ikbm8Z1WCRhulhZchEz51pFeKNBnRmtGpDPOuH wShszSNZtS6hKGMYA0i18/MQVrSfkYtljpcAvoC+6UeA+DtfF4C/VDA2B1478h3Ll3iWWEKSokv 9S8CjXmEXbXnLoc928N1VYLdLIYLaMK1K13gCQxLGhimD5pH3DiLQc4cNc/oLHtt1J/NIZ3XA3+ A= X-Received: by 2002:a1c:750a:0:b0:3f4:2bce:eef8 with SMTP id o10-20020a1c750a000000b003f42bceeef8mr19346756wmc.16.1684345530076; Wed, 17 May 2023 10:45:30 -0700 (PDT) X-Google-Smtp-Source: ACHHUZ4CQT7dIAZfym1JfMtFnZpmD20V6UFfDrKNwPIREmT9oD69qGEp2A3HkRQg1NVktn9uPxi+Cg== X-Received: by 2002:a1c:750a:0:b0:3f4:2bce:eef8 with SMTP id o10-20020a1c750a000000b003f42bceeef8mr19346739wmc.16.1684345529620; Wed, 17 May 2023 10:45:29 -0700 (PDT) Received: from [192.168.10.118] ([2001:b07:6468:f312:9af8:e5f5:7516:fa89]) by smtp.gmail.com with ESMTPSA id f26-20020a1c6a1a000000b003f4272c2d0csm2868598wmc.36.2023.05.17.10.45.28 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 17 May 2023 10:45:28 -0700 (PDT) From: Paolo Bonzini To: qemu-devel@nongnu.org Cc: Xinyu Li , qemu-stable@nongnu.org Subject: [PULL 04/68] target/i386: fix avx2 instructions vzeroall and vpermdq Date: Wed, 17 May 2023 19:44:16 +0200 Message-Id: <20230517174520.887405-5-pbonzini@redhat.com> X-Mailer: git-send-email 2.40.1 In-Reply-To: <20230517174520.887405-1-pbonzini@redhat.com> References: <20230517174520.887405-1-pbonzini@redhat.com> MIME-Version: 1.0 Received-SPF: pass client-ip=170.10.129.124; envelope-from=pbonzini@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=unavailable autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org From: Xinyu Li vzeroall: xmm_regs should be used instead of xmm_t0 vpermdq: bit 3 and 7 of imm should be considered Signed-off-by: Xinyu Li Message-Id: <20230510145222.586487-1-lixinyu20s@ict.ac.cn> Cc: qemu-stable@nongnu.org Signed-off-by: Paolo Bonzini --- target/i386/ops_sse.h | 8 ++++++++ target/i386/tcg/emit.c.inc | 2 +- 2 files changed, 9 insertions(+), 1 deletion(-) diff --git a/target/i386/ops_sse.h b/target/i386/ops_sse.h index 0bd6bfad8a17..fb63af7afa21 100644 --- a/target/i386/ops_sse.h +++ b/target/i386/ops_sse.h @@ -2497,6 +2497,14 @@ void helper_vpermdq_ymm(Reg *d, Reg *v, Reg *s, uint32_t order) d->Q(1) = r1; d->Q(2) = r2; d->Q(3) = r3; + if (order & 0x8) { + d->Q(0) = 0; + d->Q(1) = 0; + } + if (order & 0x80) { + d->Q(2) = 0; + d->Q(3) = 0; + } } void helper_vpermq_ymm(Reg *d, Reg *s, uint32_t order) diff --git a/target/i386/tcg/emit.c.inc b/target/i386/tcg/emit.c.inc index 95fb4f52fabe..4fe8dec42746 100644 --- a/target/i386/tcg/emit.c.inc +++ b/target/i386/tcg/emit.c.inc @@ -2285,7 +2285,7 @@ static void gen_VZEROALL(DisasContext *s, CPUX86State *env, X86DecodedInsn *deco { TCGv_ptr ptr = tcg_temp_new_ptr(); - tcg_gen_addi_ptr(ptr, cpu_env, offsetof(CPUX86State, xmm_t0)); + tcg_gen_addi_ptr(ptr, cpu_env, offsetof(CPUX86State, xmm_regs)); gen_helper_memset(ptr, ptr, tcg_constant_i32(0), tcg_constant_ptr(CPU_NB_REGS * sizeof(ZMMReg))); } From patchwork Wed May 17 17:44:17 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Paolo Bonzini X-Patchwork-Id: 1782847 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=nongnu.org (client-ip=209.51.188.17; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=) Authentication-Results: legolas.ozlabs.org; dkim=pass (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=ePbc/7o5; dkim-atps=neutral Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-ECDSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4QM0yw0rlLz20KF for ; Thu, 18 May 2023 03:51:20 +1000 (AEST) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1pzLDZ-0006zl-EK; Wed, 17 May 2023 13:45:41 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pzLDV-0006sy-FO for qemu-devel@nongnu.org; Wed, 17 May 2023 13:45:37 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.129.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pzLDT-00040o-RD for qemu-devel@nongnu.org; Wed, 17 May 2023 13:45:37 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1684345535; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=s3Rv+ESo7WNW5A9ZevmKZ3JyG2kyStdtRscAL+/MXEo=; b=ePbc/7o5uPdJwdiaufTVH4hKF3CG//kzJylA9onq5sj6ZR/1/xzXuFcoKwkSiyUqW2F00X 2i2GJzrNxt7RvI5nM8DEVkG4v7eKJ3JYqqTLgJE6h8LUF0F6/QAK8JWyiu0r+6GxID8/qu 7lP/IwdL2AUHSy9yt5AYZzRHcpmWRGg= Received: from mail-wr1-f70.google.com (mail-wr1-f70.google.com [209.85.221.70]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-29-ztM8WjMzP2KFx3fIHdvmbg-1; Wed, 17 May 2023 13:45:33 -0400 X-MC-Unique: ztM8WjMzP2KFx3fIHdvmbg-1 Received: by mail-wr1-f70.google.com with SMTP id ffacd0b85a97d-3063287f1c9so490262f8f.2 for ; Wed, 17 May 2023 10:45:32 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1684345531; x=1686937531; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=s3Rv+ESo7WNW5A9ZevmKZ3JyG2kyStdtRscAL+/MXEo=; b=JFMLjJWWKMBvfAtUQEvum3avg/Mm/Wd+EmFG3Q44Q81IcOjZGPjrbGAZ5hqDik871g QaVXznLMbtlHtHUJTIQpHeD72JQIK5UftG/4IgbcXfdA+NGvMu3HOY1lL+7VhR6XY7ea /oxv62CowoDmj+svgOGmdTB+JuaonGKB7UA6HiuW/cMj525VXDjT/RAJMfl773fvRt9y pRzBpE5j7O4Ht5+Oi1K/kmV9a8eV85fURvwCG5N4M6BwcvPu+lWPrmLnDamG7JjwXs2c irKtYvsgxUgj3BqjYqMNhcMAAZCWJYi2Y8AZyLh1v1s8CbGhEn8dt+CpaxCmStVp12NV Kxzw== X-Gm-Message-State: AC+VfDwCR3xI0aPWqqX3eSJaHLNGukLHYOluRO5OsxMmhY32xCuj5jC8 ReGbwfeYng9KvdvxrSh7Z0oQYOfDfvDmCItMP2mwjt0eZtfyMcKuR+fihnYY77eLM7zHFknnFDx YYDh8r4n3ml03mzjHYlQFmFq2iXXtDJVdWIF0fcrzIhOMpYTEVtae7ZM034UhW2ERsij6tU//MX w= X-Received: by 2002:a5d:6b4c:0:b0:306:339b:684 with SMTP id x12-20020a5d6b4c000000b00306339b0684mr1178716wrw.65.1684345531681; Wed, 17 May 2023 10:45:31 -0700 (PDT) X-Google-Smtp-Source: ACHHUZ6atJmOk4XTZFh9HYGm14KZMk8YC0NzuLD7kknNU6lBi80SiTvKQbci6Rtloi9hbSN68USyiw== X-Received: by 2002:a5d:6b4c:0:b0:306:339b:684 with SMTP id x12-20020a5d6b4c000000b00306339b0684mr1178704wrw.65.1684345531335; Wed, 17 May 2023 10:45:31 -0700 (PDT) Received: from [192.168.10.118] ([2001:b07:6468:f312:9af8:e5f5:7516:fa89]) by smtp.gmail.com with ESMTPSA id m16-20020a7bce10000000b003f427687ba7sm2920254wmc.41.2023.05.17.10.45.30 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 17 May 2023 10:45:30 -0700 (PDT) From: Paolo Bonzini To: qemu-devel@nongnu.org Subject: [PULL 05/68] tests/tcg/i386: correct mask for VPERM2F128/VPERM2I128 Date: Wed, 17 May 2023 19:44:17 +0200 Message-Id: <20230517174520.887405-6-pbonzini@redhat.com> X-Mailer: git-send-email 2.40.1 In-Reply-To: <20230517174520.887405-1-pbonzini@redhat.com> References: <20230517174520.887405-1-pbonzini@redhat.com> MIME-Version: 1.0 Received-SPF: pass client-ip=170.10.129.124; envelope-from=pbonzini@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org The instructions also use bits 3 and 7 of their 8-byte immediate. Signed-off-by: Paolo Bonzini --- tests/tcg/i386/test-avx.py | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/tests/tcg/i386/test-avx.py b/tests/tcg/i386/test-avx.py index d9ca00a49e6c..641a2ef69ebc 100755 --- a/tests/tcg/i386/test-avx.py +++ b/tests/tcg/i386/test-avx.py @@ -49,7 +49,7 @@ 'VEXTRACT[FI]128': 0x01, 'VINSERT[FI]128': 0x01, 'VPBLENDD': 0xff, - 'VPERM2[FI]128': 0x33, + 'VPERM2[FI]128': 0xbb, 'VPERMPD': 0xff, 'VPERMQ': 0xff, 'VPERMILPS': 0xff, From patchwork Wed May 17 17:44:18 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Paolo Bonzini X-Patchwork-Id: 1782824 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=nongnu.org (client-ip=209.51.188.17; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=) Authentication-Results: legolas.ozlabs.org; dkim=pass (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=OIBjjjO9; dkim-atps=neutral Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-ECDSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4QM0rh3Tqzz20dn for ; Thu, 18 May 2023 03:45:56 +1000 (AEST) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1pzLDa-00070Z-5J; Wed, 17 May 2023 13:45:42 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pzLDY-0006xj-9k for qemu-devel@nongnu.org; Wed, 17 May 2023 13:45:40 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pzLDV-00041M-HL for qemu-devel@nongnu.org; Wed, 17 May 2023 13:45:39 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1684345537; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=mhnQyFkEi7XQRjxdO590qYoQQBcg/WwkKHz6f3W8ubI=; b=OIBjjjO93MpCUr+2S7/wv+U63JmEg60v1gkpFZ+qZQgHrzIhFUubjPtV71ShVSqbraccTy dXgLi8sohqlu42lk4mxNS9N1u4EerKRJgYDeDV2H7/mwcweTisOdkIkjFOstU6+RkSE3a3 mdy9ddLxh6triRVBehfcheTvjdkK04g= Received: from mail-wm1-f69.google.com (mail-wm1-f69.google.com [209.85.128.69]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-16-nJ5WhI6OO0um6GeDsmn6Zg-1; Wed, 17 May 2023 13:45:35 -0400 X-MC-Unique: nJ5WhI6OO0um6GeDsmn6Zg-1 Received: by mail-wm1-f69.google.com with SMTP id 5b1f17b1804b1-3f4245ffbb4so6568535e9.3 for ; Wed, 17 May 2023 10:45:34 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1684345533; x=1686937533; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=mhnQyFkEi7XQRjxdO590qYoQQBcg/WwkKHz6f3W8ubI=; b=At/FTyDEAFt2wvwojyFFAqJo2dnTxcBhU7VqNQvtCxOA9Nn8hdklfbqiHIgej35wLa ttYnkYi3e5gfXGzgLYYcDfja/zZ6Fw2StMlk9yG1s4SpppJh4b86KSih9ixNsiDoD+Hx 1aTWB1YFW5RO7vB/fvYYcVEzhZhDX6kNz9xzDdp8oBP0ANey7dYNKhfCSXDtNomyJbsZ eJZgU0NEj1E4tUw0UHu8JQllU7mGxhHwH39X22L8kxVfWAzVTWplr5jKOSMcD4mxSKV0 +3Ru6L84wveIiz+h22+8sckVoBVZqIVelyRa7kfobpqNRDYmwCOOSzOxm/mdfWQrW90H vIpA== X-Gm-Message-State: AC+VfDxNrpZpttVwNvaWv57v8v17FdCB1Ki7XpnDicEjhMcW2KqXWioT 8p4w0txbVUer0D91tEmNokhO2915ZBMrh35cWwNdi9nmFEuvX12GbHplio1HK3EGcCfrcnKxrt0 ewHWo0gRnrPZEjqLCyJfEOang9g5L+1TqHMaOszyW3JzJaFtqcJh/jfVMyHbGomAcSEeSBQ36VP A= X-Received: by 2002:a7b:c3c9:0:b0:3f4:27db:d with SMTP id t9-20020a7bc3c9000000b003f427db000dmr26129130wmj.17.1684345533105; Wed, 17 May 2023 10:45:33 -0700 (PDT) X-Google-Smtp-Source: ACHHUZ59QLteNdC587flmTqsEP56sKyqQHBMp92VGo9OPzps2kIxYgeGAjSnmMAFUIBhSkY77kVYGA== X-Received: by 2002:a7b:c3c9:0:b0:3f4:27db:d with SMTP id t9-20020a7bc3c9000000b003f427db000dmr26129117wmj.17.1684345532801; Wed, 17 May 2023 10:45:32 -0700 (PDT) Received: from [192.168.10.118] ([2001:b07:6468:f312:9af8:e5f5:7516:fa89]) by smtp.gmail.com with ESMTPSA id o14-20020a5d58ce000000b0030629536e64sm3383554wrf.30.2023.05.17.10.45.31 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 17 May 2023 10:45:32 -0700 (PDT) From: Paolo Bonzini To: qemu-devel@nongnu.org Cc: Ricky Zhou Subject: [PULL 06/68] target/i386: Fix and add some comments next to SSE/AVX instructions. Date: Wed, 17 May 2023 19:44:18 +0200 Message-Id: <20230517174520.887405-7-pbonzini@redhat.com> X-Mailer: git-send-email 2.40.1 In-Reply-To: <20230517174520.887405-1-pbonzini@redhat.com> References: <20230517174520.887405-1-pbonzini@redhat.com> MIME-Version: 1.0 Received-SPF: pass client-ip=170.10.133.124; envelope-from=pbonzini@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org From: Ricky Zhou Adds some comments describing what instructions correspond to decoding table entries and fixes some existing comments which named the wrong instruction. Message-Id: <20230501111428.95998-1-ricky@rzhou.org> Signed-off-by: Paolo Bonzini --- target/i386/tcg/decode-new.c.inc | 24 ++++++++++++------------ 1 file changed, 12 insertions(+), 12 deletions(-) diff --git a/target/i386/tcg/decode-new.c.inc b/target/i386/tcg/decode-new.c.inc index 48fefaffdf63..e0f25e7fd8ed 100644 --- a/target/i386/tcg/decode-new.c.inc +++ b/target/i386/tcg/decode-new.c.inc @@ -274,9 +274,9 @@ static void decode_0F78(DisasContext *s, CPUX86State *env, X86OpEntry *entry, ui { static const X86OpEntry opcodes_0F78[4] = { {}, - X86_OP_ENTRY3(EXTRQ_i, V,x, None,None, I,w, cpuid(SSE4A)), + X86_OP_ENTRY3(EXTRQ_i, V,x, None,None, I,w, cpuid(SSE4A)), /* AMD extension */ {}, - X86_OP_ENTRY3(INSERTQ_i, V,x, U,x, I,w, cpuid(SSE4A)), + X86_OP_ENTRY3(INSERTQ_i, V,x, U,x, I,w, cpuid(SSE4A)), /* AMD extension */ }; *entry = *decode_by_prefix(s, opcodes_0F78); } @@ -284,9 +284,9 @@ static void decode_0F78(DisasContext *s, CPUX86State *env, X86OpEntry *entry, ui static void decode_0F79(DisasContext *s, CPUX86State *env, X86OpEntry *entry, uint8_t *b) { if (s->prefix & PREFIX_REPNZ) { - entry->gen = gen_INSERTQ_r; + entry->gen = gen_INSERTQ_r; /* AMD extension */ } else if (s->prefix & PREFIX_DATA) { - entry->gen = gen_EXTRQ_r; + entry->gen = gen_EXTRQ_r; /* AMD extension */ } else { entry->gen = NULL; }; @@ -660,15 +660,15 @@ static void decode_0F10(DisasContext *s, CPUX86State *env, X86OpEntry *entry, ui static void decode_0F11(DisasContext *s, CPUX86State *env, X86OpEntry *entry, uint8_t *b) { static const X86OpEntry opcodes_0F11_reg[4] = { - X86_OP_ENTRY3(MOVDQ, W,x, None,None, V,x, vex4), /* MOVPS */ - X86_OP_ENTRY3(MOVDQ, W,x, None,None, V,x, vex4), /* MOVPD */ + X86_OP_ENTRY3(MOVDQ, W,x, None,None, V,x, vex4), /* MOVUPS */ + X86_OP_ENTRY3(MOVDQ, W,x, None,None, V,x, vex4), /* MOVUPD */ X86_OP_ENTRY3(VMOVSS, W,x, H,x, V,x, vex4), X86_OP_ENTRY3(VMOVLPx, W,x, H,x, V,q, vex4), /* MOVSD */ }; static const X86OpEntry opcodes_0F11_mem[4] = { - X86_OP_ENTRY3(MOVDQ, W,x, None,None, V,x, vex4), /* MOVPS */ - X86_OP_ENTRY3(MOVDQ, W,x, None,None, V,x, vex4), /* MOVPD */ + X86_OP_ENTRY3(MOVDQ, W,x, None,None, V,x, vex4), /* MOVUPS */ + X86_OP_ENTRY3(MOVDQ, W,x, None,None, V,x, vex4), /* MOVUPD */ X86_OP_ENTRY3(VMOVSS_st, M,ss, None,None, V,x, vex4), X86_OP_ENTRY3(VMOVLPx_st, M,sd, None,None, V,x, vex4), /* MOVSD */ }; @@ -850,9 +850,9 @@ static const X86OpEntry opcodes_0F[256] = { [0x17] = X86_OP_ENTRY3(VMOVHPx_st, M,q, None,None, V,dq, vex4 p_00_66), [0x50] = X86_OP_ENTRY3(MOVMSK, G,y, None,None, U,x, vex7 p_00_66), - [0x51] = X86_OP_GROUP3(sse_unary, V,x, H,x, W,x, vex2_rep3 p_00_66_f3_f2), - [0x52] = X86_OP_GROUP3(sse_unary, V,x, H,x, W,x, vex4_rep5 p_00_f3), - [0x53] = X86_OP_GROUP3(sse_unary, V,x, H,x, W,x, vex4_rep5 p_00_f3), + [0x51] = X86_OP_GROUP3(sse_unary, V,x, H,x, W,x, vex2_rep3 p_00_66_f3_f2), /* sqrtps */ + [0x52] = X86_OP_GROUP3(sse_unary, V,x, H,x, W,x, vex4_rep5 p_00_f3), /* rsqrtps */ + [0x53] = X86_OP_GROUP3(sse_unary, V,x, H,x, W,x, vex4_rep5 p_00_f3), /* rcpps */ [0x54] = X86_OP_ENTRY3(PAND, V,x, H,x, W,x, vex4 p_00_66), /* vand */ [0x55] = X86_OP_ENTRY3(PANDN, V,x, H,x, W,x, vex4 p_00_66), /* vandn */ [0x56] = X86_OP_ENTRY3(POR, V,x, H,x, W,x, vex4 p_00_66), /* vor */ @@ -890,7 +890,7 @@ static const X86OpEntry opcodes_0F[256] = { [0x58] = X86_OP_ENTRY3(VADD, V,x, H,x, W,x, vex2_rep3 p_00_66_f3_f2), [0x59] = X86_OP_ENTRY3(VMUL, V,x, H,x, W,x, vex2_rep3 p_00_66_f3_f2), - [0x5a] = X86_OP_GROUP3(sse_unary, V,x, H,x, W,x, vex2_rep3 p_00_66_f3_f2), + [0x5a] = X86_OP_GROUP3(sse_unary, V,x, H,x, W,x, vex2_rep3 p_00_66_f3_f2), /* CVTPS2PD */ [0x5b] = X86_OP_GROUP0(0F5B), [0x5c] = X86_OP_ENTRY3(VSUB, V,x, H,x, W,x, vex2_rep3 p_00_66_f3_f2), [0x5d] = X86_OP_ENTRY3(VMIN, V,x, H,x, W,x, vex2_rep3 p_00_66_f3_f2), From patchwork Wed May 17 17:44:19 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Paolo Bonzini X-Patchwork-Id: 1782827 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=nongnu.org (client-ip=209.51.188.17; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=) Authentication-Results: legolas.ozlabs.org; dkim=pass (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=DD+65PJh; dkim-atps=neutral Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-ECDSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4QM0s94bxgz20dq for ; Thu, 18 May 2023 03:46:21 +1000 (AEST) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1pzLDj-00075f-NY; Wed, 17 May 2023 13:45:51 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pzLDb-000727-M8 for qemu-devel@nongnu.org; Wed, 17 May 2023 13:45:43 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.129.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pzLDY-00041j-Jv for qemu-devel@nongnu.org; Wed, 17 May 2023 13:45:43 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1684345538; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=M+LJ4nMmE0UUrO9wa4oiwOT/ScyWbCv2g8woZUrOBQQ=; b=DD+65PJhpx11k0s8ijBiEQvVTZYokvW1ndZqWjkomgZ6/7WuN8aU5zbWCRr6zAJS/nQxCN KGszOQQe2q9fJFAKMHq23G9iCkfmbkrOB3ecuiL5GuggM+GRifVFHoDhxEmL3XhEHU8LwX mV5cvd3kcw8J3s01S3YNFHkzhsRT1l4= Received: from mail-wm1-f71.google.com (mail-wm1-f71.google.com [209.85.128.71]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-103-c-RbPZFQONaAfBtnXBR8jA-1; Wed, 17 May 2023 13:45:37 -0400 X-MC-Unique: c-RbPZFQONaAfBtnXBR8jA-1 Received: by mail-wm1-f71.google.com with SMTP id 5b1f17b1804b1-3f421cfb4beso4797495e9.0 for ; Wed, 17 May 2023 10:45:36 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1684345535; x=1686937535; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=M+LJ4nMmE0UUrO9wa4oiwOT/ScyWbCv2g8woZUrOBQQ=; b=Dj/GTyelBoliH8ZapnHUvHj+uJ4LeiZhtZhSZwlQHq7R+25TnTZgXthk54djvpxo/j Roq46NhVyexvI9aWcJN/grt2pBSePSZ1sO6n7jFpKOcLj3qAiVMkD/cQ1cVqGUKQ/a5+ //UaBBxhqvIUKG9BXe6b9hvDaL6rz9Hdo/91Y2dxx5P+Cu3VobzaEEw9Tt/VylzI7hzw zLm+tQ9kAwzPP3MaSPWiroHN4r+7xFNxLrXbhcTf7LNGQ/NhprGdyz4m1tclAcDgsGYx l8QvyAEJkUgcFDw0Xk58i+mWz9oPgOq2AlePDmqRtEeYewxWlX95KRdzXoXHwpG1T/vI ouqA== X-Gm-Message-State: AC+VfDwaJQeYZ2JDDAThiydeZKnJ8Br0sGNGhzoakSdW6CRYKGs5RHIV 39qkVmpk/f6wHOaZefme7+WrjBSNxQ2mbzcqldztUTRFuH8yw+tfb3HGJ/8CIQ8UUoTgOk9++/g 1PKSf8bHHsgWtdR1y0UD6/ZeT9SzmrosMYjecwpo5/DBLk1V1Ixiodk9344k4qgk8HEMCALLysG Y= X-Received: by 2002:a05:600c:b42:b0:3f4:f0c2:125 with SMTP id k2-20020a05600c0b4200b003f4f0c20125mr14058388wmr.23.1684345534712; Wed, 17 May 2023 10:45:34 -0700 (PDT) X-Google-Smtp-Source: ACHHUZ4iLNlKhenAyeibHEILjoFWvJU/Yn+hOIx5/rHBWVfgnHydzb7U2tKmwYXDlcrlJhAhoeKLvg== X-Received: by 2002:a05:600c:b42:b0:3f4:f0c2:125 with SMTP id k2-20020a05600c0b4200b003f4f0c20125mr14058375wmr.23.1684345534351; Wed, 17 May 2023 10:45:34 -0700 (PDT) Received: from [192.168.10.118] ([2001:b07:6468:f312:9af8:e5f5:7516:fa89]) by smtp.gmail.com with ESMTPSA id a3-20020a05600c224300b003f17848673fsm2868969wmm.27.2023.05.17.10.45.33 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 17 May 2023 10:45:33 -0700 (PDT) From: Paolo Bonzini To: qemu-devel@nongnu.org Cc: Ricky Zhou Subject: [PULL 07/68] target/i386: Fix exception classes for SSE/AVX instructions. Date: Wed, 17 May 2023 19:44:19 +0200 Message-Id: <20230517174520.887405-8-pbonzini@redhat.com> X-Mailer: git-send-email 2.40.1 In-Reply-To: <20230517174520.887405-1-pbonzini@redhat.com> References: <20230517174520.887405-1-pbonzini@redhat.com> MIME-Version: 1.0 Received-SPF: pass client-ip=170.10.129.124; envelope-from=pbonzini@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org From: Ricky Zhou Fix the exception classes for some SSE/AVX instructions to match what is documented in the Intel manual. These changes are expected to have no functional effect on the behavior that qemu implements (primarily >= 16-byte memory alignment checks). For instance, since qemu does not implement the AC flag, there is no difference in behavior between Exception Classes 4 and 5 for instructions where the SSE version only takes <16 byte memory operands. Message-Id: <20230501111428.95998-2-ricky@rzhou.org> Signed-off-by: Paolo Bonzini --- target/i386/tcg/decode-new.c.inc | 46 ++++++++++++++++---------------- 1 file changed, 23 insertions(+), 23 deletions(-) diff --git a/target/i386/tcg/decode-new.c.inc b/target/i386/tcg/decode-new.c.inc index e0f25e7fd8ed..1a789201f740 100644 --- a/target/i386/tcg/decode-new.c.inc +++ b/target/i386/tcg/decode-new.c.inc @@ -237,7 +237,7 @@ static void decode_group14(DisasContext *s, CPUX86State *env, X86OpEntry *entry, static void decode_0F6F(DisasContext *s, CPUX86State *env, X86OpEntry *entry, uint8_t *b) { static const X86OpEntry opcodes_0F6F[4] = { - X86_OP_ENTRY3(MOVDQ, P,q, None,None, Q,q, vex1 mmx), /* movq */ + X86_OP_ENTRY3(MOVDQ, P,q, None,None, Q,q, vex5 mmx), /* movq */ X86_OP_ENTRY3(MOVDQ, V,x, None,None, W,x, vex1), /* movdqa */ X86_OP_ENTRY3(MOVDQ, V,x, None,None, W,x, vex4_unal), /* movdqu */ {}, @@ -306,7 +306,7 @@ static void decode_0F7E(DisasContext *s, CPUX86State *env, X86OpEntry *entry, ui static void decode_0F7F(DisasContext *s, CPUX86State *env, X86OpEntry *entry, uint8_t *b) { static const X86OpEntry opcodes_0F7F[4] = { - X86_OP_ENTRY3(MOVDQ, W,x, None,None, V,x, vex1 mmx), /* movq */ + X86_OP_ENTRY3(MOVDQ, W,x, None,None, V,x, vex5 mmx), /* movq */ X86_OP_ENTRY3(MOVDQ, W,x, None,None, V,x, vex1), /* movdqa */ X86_OP_ENTRY3(MOVDQ, W,x, None,None, V,x, vex4_unal), /* movdqu */ {}, @@ -639,15 +639,15 @@ static void decode_0F10(DisasContext *s, CPUX86State *env, X86OpEntry *entry, ui static const X86OpEntry opcodes_0F10_reg[4] = { X86_OP_ENTRY3(MOVDQ, V,x, None,None, W,x, vex4_unal), /* MOVUPS */ X86_OP_ENTRY3(MOVDQ, V,x, None,None, W,x, vex4_unal), /* MOVUPD */ - X86_OP_ENTRY3(VMOVSS, V,x, H,x, W,x, vex4), - X86_OP_ENTRY3(VMOVLPx, V,x, H,x, W,x, vex4), /* MOVSD */ + X86_OP_ENTRY3(VMOVSS, V,x, H,x, W,x, vex5), + X86_OP_ENTRY3(VMOVLPx, V,x, H,x, W,x, vex5), /* MOVSD */ }; static const X86OpEntry opcodes_0F10_mem[4] = { X86_OP_ENTRY3(MOVDQ, V,x, None,None, W,x, vex4_unal), /* MOVUPS */ X86_OP_ENTRY3(MOVDQ, V,x, None,None, W,x, vex4_unal), /* MOVUPD */ - X86_OP_ENTRY3(VMOVSS_ld, V,x, H,x, M,ss, vex4), - X86_OP_ENTRY3(VMOVSD_ld, V,x, H,x, M,sd, vex4), + X86_OP_ENTRY3(VMOVSS_ld, V,x, H,x, M,ss, vex5), + X86_OP_ENTRY3(VMOVSD_ld, V,x, H,x, M,sd, vex5), }; if ((get_modrm(s, env) >> 6) == 3) { @@ -662,15 +662,15 @@ static void decode_0F11(DisasContext *s, CPUX86State *env, X86OpEntry *entry, ui static const X86OpEntry opcodes_0F11_reg[4] = { X86_OP_ENTRY3(MOVDQ, W,x, None,None, V,x, vex4), /* MOVUPS */ X86_OP_ENTRY3(MOVDQ, W,x, None,None, V,x, vex4), /* MOVUPD */ - X86_OP_ENTRY3(VMOVSS, W,x, H,x, V,x, vex4), - X86_OP_ENTRY3(VMOVLPx, W,x, H,x, V,q, vex4), /* MOVSD */ + X86_OP_ENTRY3(VMOVSS, W,x, H,x, V,x, vex5), + X86_OP_ENTRY3(VMOVLPx, W,x, H,x, V,q, vex5), /* MOVSD */ }; static const X86OpEntry opcodes_0F11_mem[4] = { X86_OP_ENTRY3(MOVDQ, W,x, None,None, V,x, vex4), /* MOVUPS */ X86_OP_ENTRY3(MOVDQ, W,x, None,None, V,x, vex4), /* MOVUPD */ - X86_OP_ENTRY3(VMOVSS_st, M,ss, None,None, V,x, vex4), - X86_OP_ENTRY3(VMOVLPx_st, M,sd, None,None, V,x, vex4), /* MOVSD */ + X86_OP_ENTRY3(VMOVSS_st, M,ss, None,None, V,x, vex5), + X86_OP_ENTRY3(VMOVLPx_st, M,sd, None,None, V,x, vex5), /* MOVSD */ }; if ((get_modrm(s, env) >> 6) == 3) { @@ -687,16 +687,16 @@ static void decode_0F12(DisasContext *s, CPUX86State *env, X86OpEntry *entry, ui * Use dq for operand for compatibility with gen_MOVSD and * to allow VEX128 only. */ - X86_OP_ENTRY3(VMOVLPx_ld, V,dq, H,dq, M,q, vex4), /* MOVLPS */ - X86_OP_ENTRY3(VMOVLPx_ld, V,dq, H,dq, M,q, vex4), /* MOVLPD */ + X86_OP_ENTRY3(VMOVLPx_ld, V,dq, H,dq, M,q, vex5), /* MOVLPS */ + X86_OP_ENTRY3(VMOVLPx_ld, V,dq, H,dq, M,q, vex5), /* MOVLPD */ X86_OP_ENTRY3(VMOVSLDUP, V,x, None,None, W,x, vex4 cpuid(SSE3)), - X86_OP_ENTRY3(VMOVDDUP, V,x, None,None, WM,q, vex4 cpuid(SSE3)), /* qq if VEX.256 */ + X86_OP_ENTRY3(VMOVDDUP, V,x, None,None, WM,q, vex5 cpuid(SSE3)), /* qq if VEX.256 */ }; static const X86OpEntry opcodes_0F12_reg[4] = { - X86_OP_ENTRY3(VMOVHLPS, V,dq, H,dq, U,dq, vex4), - X86_OP_ENTRY3(VMOVLPx, W,x, H,x, U,q, vex4), /* MOVLPD */ + X86_OP_ENTRY3(VMOVHLPS, V,dq, H,dq, U,dq, vex7), + X86_OP_ENTRY3(VMOVLPx, W,x, H,x, U,q, vex5), /* MOVLPD */ X86_OP_ENTRY3(VMOVSLDUP, V,x, None,None, U,x, vex4 cpuid(SSE3)), - X86_OP_ENTRY3(VMOVDDUP, V,x, None,None, U,x, vex4 cpuid(SSE3)), + X86_OP_ENTRY3(VMOVDDUP, V,x, None,None, U,x, vex5 cpuid(SSE3)), }; if ((get_modrm(s, env) >> 6) == 3) { @@ -716,15 +716,15 @@ static void decode_0F16(DisasContext *s, CPUX86State *env, X86OpEntry *entry, ui * Operand 1 technically only reads the low 64 bits, but uses dq so that * it is easier to check for op0 == op1 in an endianness-neutral manner. */ - X86_OP_ENTRY3(VMOVHPx_ld, V,dq, H,dq, M,q, vex4), /* MOVHPS */ - X86_OP_ENTRY3(VMOVHPx_ld, V,dq, H,dq, M,q, vex4), /* MOVHPD */ + X86_OP_ENTRY3(VMOVHPx_ld, V,dq, H,dq, M,q, vex5), /* MOVHPS */ + X86_OP_ENTRY3(VMOVHPx_ld, V,dq, H,dq, M,q, vex5), /* MOVHPD */ X86_OP_ENTRY3(VMOVSHDUP, V,x, None,None, W,x, vex4 cpuid(SSE3)), {}, }; static const X86OpEntry opcodes_0F16_reg[4] = { /* Same as above, operand 1 could be Hq if it wasn't for big-endian. */ - X86_OP_ENTRY3(VMOVLHPS, V,dq, H,dq, U,q, vex4), - X86_OP_ENTRY3(VMOVHPx, V,x, H,x, U,x, vex4), /* MOVHPD */ + X86_OP_ENTRY3(VMOVLHPS, V,dq, H,dq, U,q, vex7), + X86_OP_ENTRY3(VMOVHPx, V,x, H,x, U,x, vex5), /* MOVHPD */ X86_OP_ENTRY3(VMOVSHDUP, V,x, None,None, U,x, vex4 cpuid(SSE3)), {}, }; @@ -824,7 +824,7 @@ static void decode_0FE6(DisasContext *s, CPUX86State *env, X86OpEntry *entry, ui static const X86OpEntry opcodes_0FE6[4] = { {}, X86_OP_ENTRY2(VCVTTPD2DQ, V,x, W,x, vex2), - X86_OP_ENTRY2(VCVTDQ2PD, V,x, W,x, vex2), + X86_OP_ENTRY2(VCVTDQ2PD, V,x, W,x, vex5), X86_OP_ENTRY2(VCVTPD2DQ, V,x, W,x, vex2), }; *entry = *decode_by_prefix(s, opcodes_0FE6); @@ -842,12 +842,12 @@ static const X86OpEntry opcodes_0F[256] = { [0x10] = X86_OP_GROUP0(0F10), [0x11] = X86_OP_GROUP0(0F11), [0x12] = X86_OP_GROUP0(0F12), - [0x13] = X86_OP_ENTRY3(VMOVLPx_st, M,q, None,None, V,q, vex4 p_00_66), + [0x13] = X86_OP_ENTRY3(VMOVLPx_st, M,q, None,None, V,q, vex5 p_00_66), [0x14] = X86_OP_ENTRY3(VUNPCKLPx, V,x, H,x, W,x, vex4 p_00_66), [0x15] = X86_OP_ENTRY3(VUNPCKHPx, V,x, H,x, W,x, vex4 p_00_66), [0x16] = X86_OP_GROUP0(0F16), /* Incorrectly listed as Mq,Vq in the manual */ - [0x17] = X86_OP_ENTRY3(VMOVHPx_st, M,q, None,None, V,dq, vex4 p_00_66), + [0x17] = X86_OP_ENTRY3(VMOVHPx_st, M,q, None,None, V,dq, vex5 p_00_66), [0x50] = X86_OP_ENTRY3(MOVMSK, G,y, None,None, U,x, vex7 p_00_66), [0x51] = X86_OP_GROUP3(sse_unary, V,x, H,x, W,x, vex2_rep3 p_00_66_f3_f2), /* sqrtps */ From patchwork Wed May 17 17:44:20 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Paolo Bonzini X-Patchwork-Id: 1782823 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=nongnu.org (client-ip=209.51.188.17; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=) Authentication-Results: legolas.ozlabs.org; dkim=pass (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=RXCExR40; dkim-atps=neutral Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-ECDSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4QM0rf5q6Yz20dn for ; Thu, 18 May 2023 03:45:54 +1000 (AEST) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1pzLDb-00072O-VH; Wed, 17 May 2023 13:45:44 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pzLDZ-00070J-TG for qemu-devel@nongnu.org; Wed, 17 May 2023 13:45:41 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pzLDY-00041t-Bc for qemu-devel@nongnu.org; Wed, 17 May 2023 13:45:41 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1684345539; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=B1pz9pdYr7Ol3IJlNK5x42XWKbQZW+4OcYJ5orhyp1E=; b=RXCExR40qDqnJ5rXQF2MkrHWe8NTdXN4oVmjlHKBgQF69C0LOJlufDxzuk9PxvW5N5BGwI B77UTI42ZX7u6Mz9tMMi4OnUSON6E01P6GUsuu9ssJ8LroTeR4SRh9bdpEiw7vUDrqeKtO j2Gl5VSEOqOSBoVSEminY52lZ1K279M= Received: from mail-wm1-f72.google.com (mail-wm1-f72.google.com [209.85.128.72]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-644-lCwAIA9mOQSS8bvQJDC3hQ-1; Wed, 17 May 2023 13:45:38 -0400 X-MC-Unique: lCwAIA9mOQSS8bvQJDC3hQ-1 Received: by mail-wm1-f72.google.com with SMTP id 5b1f17b1804b1-3f419a053c5so1702415e9.1 for ; Wed, 17 May 2023 10:45:38 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1684345537; x=1686937537; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=B1pz9pdYr7Ol3IJlNK5x42XWKbQZW+4OcYJ5orhyp1E=; b=IEy2vUmdpnP7i4bIg7Cn/W5+Gr+jA9PLoBJLChAiYQJrc0IF7P0aiEMmFh47wmNzJB OOlduCFl4+tbpgXSKOdzQW2LU/3RlgiV8szUIaWXILtL5iCmIVqQ2zeL44ROSuc64/A6 KgoYdJKmDJ43hQh3/+/53z0jZWlDYhXxcJYd40vTJXVqWKkSxf5tSfhQlabG9BY7SuDX YBjknXYhG1EZXtPPxNkdMF05Lbvok0pqeLGLHPs9Ablx7S+1kHxvQTGOefRL1yG0lGL7 tGkhf/M9K3yR9XVtb9q4XgUwQyfTonUUOJZ/afKl+dEBeBGrP8i9/a0jUHTH+KWad8X2 1u3g== X-Gm-Message-State: AC+VfDwr5QwKKKI7b0wiFg+0mUuVKKM3LS87QW+31TkJ90PVNIcSwvgL MkIMjQz0Rr6BOabxe+3KyYs3bGMV1tkT36ejscS5F539qJXo2DAj/+vyt7bbNel5KUQxh0oONXv xFm8SxrekhVj6QHuOmjqacWV+RbhDH/Hn5DtPhLU4oRIhkhm4I/gXd2kE3SS21krytx5GLBFDVa 8= X-Received: by 2002:a1c:cc12:0:b0:3f5:e29:bcde with SMTP id h18-20020a1ccc12000000b003f50e29bcdemr2187519wmb.17.1684345536843; Wed, 17 May 2023 10:45:36 -0700 (PDT) X-Google-Smtp-Source: ACHHUZ4xxfQvWStseqcwO5EIP6FJshv5wC/azrrHcS1voG3xNuGoxsIZ0Ec2ptsTuY3olXLMYDT0Gw== X-Received: by 2002:a1c:cc12:0:b0:3f5:e29:bcde with SMTP id h18-20020a1ccc12000000b003f50e29bcdemr2187507wmb.17.1684345536432; Wed, 17 May 2023 10:45:36 -0700 (PDT) Received: from [192.168.10.118] ([2001:b07:6468:f312:9af8:e5f5:7516:fa89]) by smtp.gmail.com with ESMTPSA id n16-20020a1c7210000000b003f19b3d89e9sm2802768wmc.33.2023.05.17.10.45.35 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 17 May 2023 10:45:35 -0700 (PDT) From: Paolo Bonzini To: qemu-devel@nongnu.org Cc: Ricky Zhou Subject: [PULL 08/68] target/i386: Fix exception classes for MOVNTPS/MOVNTPD. Date: Wed, 17 May 2023 19:44:20 +0200 Message-Id: <20230517174520.887405-9-pbonzini@redhat.com> X-Mailer: git-send-email 2.40.1 In-Reply-To: <20230517174520.887405-1-pbonzini@redhat.com> References: <20230517174520.887405-1-pbonzini@redhat.com> MIME-Version: 1.0 Received-SPF: pass client-ip=170.10.133.124; envelope-from=pbonzini@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org From: Ricky Zhou Before this change, MOVNTPS and MOVNTPD were labeled as Exception Class 4 (only requiring alignment for legacy SSE instructions). This changes them to Exception Class 1 (always requiring memory alignment), as documented in the Intel manual. Message-Id: <20230501111428.95998-3-ricky@rzhou.org> Signed-off-by: Paolo Bonzini --- target/i386/tcg/decode-new.c.inc | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) diff --git a/target/i386/tcg/decode-new.c.inc b/target/i386/tcg/decode-new.c.inc index 1a789201f740..46afd9960bb1 100644 --- a/target/i386/tcg/decode-new.c.inc +++ b/target/i386/tcg/decode-new.c.inc @@ -750,8 +750,9 @@ static void decode_0F2A(DisasContext *s, CPUX86State *env, X86OpEntry *entry, ui static void decode_0F2B(DisasContext *s, CPUX86State *env, X86OpEntry *entry, uint8_t *b) { static const X86OpEntry opcodes_0F2B[4] = { - X86_OP_ENTRY3(MOVDQ, M,x, None,None, V,x, vex4), /* MOVNTPS */ - X86_OP_ENTRY3(MOVDQ, M,x, None,None, V,x, vex4), /* MOVNTPD */ + X86_OP_ENTRY3(MOVDQ, M,x, None,None, V,x, vex1), /* MOVNTPS */ + X86_OP_ENTRY3(MOVDQ, M,x, None,None, V,x, vex1), /* MOVNTPD */ + /* AMD extensions */ X86_OP_ENTRY3(VMOVSS_st, M,ss, None,None, V,x, vex4 cpuid(SSE4A)), /* MOVNTSS */ X86_OP_ENTRY3(VMOVLPx_st, M,sd, None,None, V,x, vex4 cpuid(SSE4A)), /* MOVNTSD */ }; From patchwork Wed May 17 17:44:21 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Paolo Bonzini X-Patchwork-Id: 1782825 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=nongnu.org (client-ip=209.51.188.17; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=) Authentication-Results: legolas.ozlabs.org; dkim=pass (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=dUEjk19y; dkim-atps=neutral Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-ECDSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4QM0rq6NJWz20dn for ; Thu, 18 May 2023 03:46:03 +1000 (AEST) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1pzLDj-00076K-Sl; Wed, 17 May 2023 13:45:51 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pzLDb-00072A-PA for qemu-devel@nongnu.org; Wed, 17 May 2023 13:45:43 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.129.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pzLDZ-00042M-SW for qemu-devel@nongnu.org; Wed, 17 May 2023 13:45:43 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1684345541; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=9VpYpqcmx3CeiKdheVcW1c/Ecj3zfYd8vAZWtX94+BU=; b=dUEjk19yhnV8UFW4RxiRln1Vi5ooEW1JVkB9SQNkn9kdyrakT53HQdrLWjKWghfu10ceZt MyUvTYLmzX6OufsGwUqqi+d6s6Leii70xD02NMnMQW0CGzQgcLg0rZnaeKlvFynnwXedCV bu8hhnvfNWMTVVrgnzD97tJQpNmjX4Y= Received: from mail-wm1-f69.google.com (mail-wm1-f69.google.com [209.85.128.69]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-483-jMsXP-hiMxa9k5e3WSVVyw-1; Wed, 17 May 2023 13:45:40 -0400 X-MC-Unique: jMsXP-hiMxa9k5e3WSVVyw-1 Received: by mail-wm1-f69.google.com with SMTP id 5b1f17b1804b1-3f420ec766dso7589935e9.1 for ; Wed, 17 May 2023 10:45:39 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1684345538; x=1686937538; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=9VpYpqcmx3CeiKdheVcW1c/Ecj3zfYd8vAZWtX94+BU=; b=G36wzoyc1AR5CKRDBIEtfIL51Qvv5v0aP+CTxg+YTSRuQfLsATZPKuN1QfyGTjfGS9 Xb8EiF+UZJtjNzrjpVrdEXkJCRXZBa8Uk0Al5S+6txf2JmTkImBrfDd+Yn0Q3H02b/4+ M51++IusIc6xHXCdy3Kbli7GSrRZ3gba2VrAQb5wtFq7bE6In/OPLXKEySQ6S6kOHvs5 YQOnhTIWnmpiEfovK09KiSpxEwGX+PIOGybQcE5tjn1xRxhnrSj9S9uMD+LrG4nJVfut lT0FfeNUTWHbVclxGlhVXYtHShPDs25zZ3bPt1Pq93oBH7z8VmJkwTFNIiZw1ZH2QjF4 pL8Q== X-Gm-Message-State: AC+VfDygL79bxKdmyxZ4ahTdRkzZoilvs3B4WJBfQr6UjonzsvUZUi/q QBcOOqTmDwWAXWC5tyzT8qSoNkfqTAjehOe8F6Nl5TlvUUuE5E0JgGALteW/MQH397NiDZMLgzo X6wAHn4PcUlyFHFRE1rYgTQPJXamjAIW9JJmHaaKZ3+9sYsh6rs0kHMIYb2DffYT3997uE+HPUd 4= X-Received: by 2002:a5d:4e0e:0:b0:309:11d8:62a6 with SMTP id p14-20020a5d4e0e000000b0030911d862a6mr1340862wrt.43.1684345538366; Wed, 17 May 2023 10:45:38 -0700 (PDT) X-Google-Smtp-Source: ACHHUZ4BDn6R7mUctonk/SrK9zL5U0HCxYx0VDpUYi5QN+3JekM+W5YgjUKInurRMxog9HN3aRTKLg== X-Received: by 2002:a5d:4e0e:0:b0:309:11d8:62a6 with SMTP id p14-20020a5d4e0e000000b0030911d862a6mr1340849wrt.43.1684345538037; Wed, 17 May 2023 10:45:38 -0700 (PDT) Received: from [192.168.10.118] ([2001:b07:6468:f312:9af8:e5f5:7516:fa89]) by smtp.gmail.com with ESMTPSA id c4-20020a056000104400b0030922ba6d0csm3781316wrx.45.2023.05.17.10.45.37 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 17 May 2023 10:45:37 -0700 (PDT) From: Paolo Bonzini To: qemu-devel@nongnu.org Cc: Fabiano Rosas , =?utf-8?q?Daniel_P_=2E_Berrang=C3=A9?= Subject: [PULL 09/68] meson: Pass -j option to sphinx Date: Wed, 17 May 2023 19:44:21 +0200 Message-Id: <20230517174520.887405-10-pbonzini@redhat.com> X-Mailer: git-send-email 2.40.1 In-Reply-To: <20230517174520.887405-1-pbonzini@redhat.com> References: <20230517174520.887405-1-pbonzini@redhat.com> MIME-Version: 1.0 Received-SPF: pass client-ip=170.10.129.124; envelope-from=pbonzini@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org From: Fabiano Rosas Save a bit of build time by passing the number of jobs option to sphinx. We cannot use the -j option from make because meson does not support setting build time parameters for custom targets. Use nproc instead or the equivalent sphinx option "-j auto", if that is available (version >=1.7.0). Also make sure our plugins support parallelism and report it properly to sphinx. Particularly, implement the merge_domaindata method in DBusDomain that is used to merge in data from other subprocesses. Tested-by: Daniel P. BerrangĂ© Signed-off-by: Fabiano Rosas Message-Id: <20230503203947.3417-2-farosas@suse.de> Signed-off-by: Paolo Bonzini --- docs/meson.build | 12 ++++++++++++ docs/sphinx/dbusdomain.py | 4 ++++ docs/sphinx/fakedbusdoc.py | 5 +++++ docs/sphinx/qmp_lexer.py | 5 +++++ 4 files changed, 26 insertions(+) diff --git a/docs/meson.build b/docs/meson.build index f220800e3e59..6d0986579e17 100644 --- a/docs/meson.build +++ b/docs/meson.build @@ -10,6 +10,18 @@ if sphinx_build.found() SPHINX_ARGS += [ '-W', '-Dkerneldoc_werror=1' ] endif + sphinx_version = run_command(SPHINX_ARGS + ['--version'], + check: true).stdout().split()[1] + if sphinx_version.version_compare('>=1.7.0') + SPHINX_ARGS += ['-j', 'auto'] + else + nproc = find_program('nproc') + if nproc.found() + jobs = run_command(nproc, check: true).stdout() + SPHINX_ARGS += ['-j', jobs] + endif + endif + # This is a bit awkward but works: create a trivial document and # try to run it with our configuration file (which enforces a # version requirement). This will fail if sphinx-build is too old. diff --git a/docs/sphinx/dbusdomain.py b/docs/sphinx/dbusdomain.py index 2ea95af623d2..9872fd5bf6a4 100644 --- a/docs/sphinx/dbusdomain.py +++ b/docs/sphinx/dbusdomain.py @@ -400,6 +400,10 @@ def get_objects(self) -> Iterator[Tuple[str, str, str, str, str, int]]: for refname, obj in self.objects.items(): yield (refname, refname, obj.objtype, obj.docname, obj.node_id, 1) + def merge_domaindata(self, docnames, otherdata): + for name, obj in otherdata['objects'].items(): + if obj.docname in docnames: + self.data['objects'][name] = obj def setup(app): app.add_domain(DBusDomain) diff --git a/docs/sphinx/fakedbusdoc.py b/docs/sphinx/fakedbusdoc.py index d2c507904654..2d2e6ef64035 100644 --- a/docs/sphinx/fakedbusdoc.py +++ b/docs/sphinx/fakedbusdoc.py @@ -23,3 +23,8 @@ def run(self): def setup(app: Sphinx) -> Dict[str, Any]: """Register a fake dbus-doc directive with Sphinx""" app.add_directive("dbus-doc", FakeDBusDocDirective) + + return dict( + parallel_read_safe = True, + parallel_write_safe = True + ) diff --git a/docs/sphinx/qmp_lexer.py b/docs/sphinx/qmp_lexer.py index f7e4c0e19860..a59de8a079c0 100644 --- a/docs/sphinx/qmp_lexer.py +++ b/docs/sphinx/qmp_lexer.py @@ -41,3 +41,8 @@ def setup(sphinx): sphinx.add_lexer('QMP', QMPExampleLexer) except errors.VersionRequirementError: sphinx.add_lexer('QMP', QMPExampleLexer()) + + return dict( + parallel_read_safe = True, + parallel_write_safe = True + ) From patchwork Wed May 17 17:44:22 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Paolo Bonzini X-Patchwork-Id: 1782829 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=nongnu.org (client-ip=209.51.188.17; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=) Authentication-Results: legolas.ozlabs.org; dkim=pass (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=bOm0t5ad; dkim-atps=neutral Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-ECDSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4QM0sd4gKRz20dn for ; Thu, 18 May 2023 03:46:45 +1000 (AEST) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1pzLDl-0007AO-Pz; Wed, 17 May 2023 13:45:55 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pzLDe-00072u-5b for qemu-devel@nongnu.org; Wed, 17 May 2023 13:45:48 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pzLDc-00042w-2z for qemu-devel@nongnu.org; Wed, 17 May 2023 13:45:45 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1684345543; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=yH+74KQ2tsoEwqM9l5YWtbigZxwIdDyrRmSJ6XlLhtY=; b=bOm0t5advQMdNAv5tdkQoF24/1XFm1ojIHebbujynFIBABG55OY6YV1HG4MDJGPI3p7jbh VTDG3pbPkeDdDXYCuGI5cG/KUFuack0lLcnkzhwEOREWQelnYZp9+pcV7rmBWVRVpTuBwd A8sJqLe6QlppDiRPVu9SxQbIGVOLx7M= Received: from mail-wm1-f72.google.com (mail-wm1-f72.google.com [209.85.128.72]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-468-m4owjuMwN_is85t91IkCmw-1; Wed, 17 May 2023 13:45:42 -0400 X-MC-Unique: m4owjuMwN_is85t91IkCmw-1 Received: by mail-wm1-f72.google.com with SMTP id 5b1f17b1804b1-3f4fb0193c6so4460635e9.1 for ; Wed, 17 May 2023 10:45:41 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1684345540; x=1686937540; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=yH+74KQ2tsoEwqM9l5YWtbigZxwIdDyrRmSJ6XlLhtY=; b=V6QFDzXeI9s7rHxfmNLvGC5ntCCYoRy7FW6AEgiG9DII7RTiiUY1EBJjprX98a6X+e EHs2Ccw4tVc6C+jP0TbD1K4jrik5u/74i+zXR/X642V+JGf2HaiBvUycwjMxmTNfVLX5 STcJe+uXvctgBLLYImYKGTdPd3PX57UHWmN9DkMZ9siCdPu/JzxahW0nmSQWaObfNQj7 rQx2Q95XixZDt73FB66vv3j6DqPGiAgHi5l+RxAurhHVayf0WwrbITWlmDD8cro7QTlj X2Xb5pLuO27JcLqHujpqIlh0Bqrf3mOtlPVriuxz2dFFosWs2HJJv2pvlGnxtG0lrD/4 jFQA== X-Gm-Message-State: AC+VfDy7RNnjxJhxeFf3eUvmNpKI/l4uGAjcDdniHfxJH7LUnyCzTwSv eEsStysK7KeudYI4QRTTjSzXrYRGx8OUkBy/2ZJx/wWzPVqM1DDbCjoCqptoXPnhVqM2g3VvOqk R1B2Oi8rEGDijRaVi1sL88yFmxph73N3t7GumoBV23M98JNM5DdJXlHLrixpBsFE4o8aD+mIor1 Y= X-Received: by 2002:a5d:4611:0:b0:309:39a0:beff with SMTP id t17-20020a5d4611000000b0030939a0beffmr1350587wrq.5.1684345540308; Wed, 17 May 2023 10:45:40 -0700 (PDT) X-Google-Smtp-Source: ACHHUZ7hwHOmROQjWA1MDza+dftzgrmXtLu0IXbENyOic6ktxjxJ0a64iSdNAAKhQ8DkUdbPXh7I0w== X-Received: by 2002:a5d:4611:0:b0:309:39a0:beff with SMTP id t17-20020a5d4611000000b0030939a0beffmr1350569wrq.5.1684345539949; Wed, 17 May 2023 10:45:39 -0700 (PDT) Received: from [192.168.10.118] ([2001:b07:6468:f312:9af8:e5f5:7516:fa89]) by smtp.gmail.com with ESMTPSA id a3-20020a05600c224300b003f17848673fsm2869135wmm.27.2023.05.17.10.45.38 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 17 May 2023 10:45:39 -0700 (PDT) From: Paolo Bonzini To: qemu-devel@nongnu.org Cc: Gavin Shan , Peter Xu , Zhenyu Zhang Subject: [PULL 10/68] migration: Add last stage indicator to global dirty log Date: Wed, 17 May 2023 19:44:22 +0200 Message-Id: <20230517174520.887405-11-pbonzini@redhat.com> X-Mailer: git-send-email 2.40.1 In-Reply-To: <20230517174520.887405-1-pbonzini@redhat.com> References: <20230517174520.887405-1-pbonzini@redhat.com> MIME-Version: 1.0 Received-SPF: pass client-ip=170.10.133.124; envelope-from=pbonzini@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org From: Gavin Shan The global dirty log synchronization is used when KVM and dirty ring are enabled. There is a particularity for ARM64 where the backup bitmap is used to track dirty pages in non-running-vcpu situations. It means the dirty ring works with the combination of ring buffer and backup bitmap. The dirty bits in the backup bitmap needs to collected in the last stage of live migration. In order to identify the last stage of live migration and pass it down, an extra parameter is added to the relevant functions and callbacks. This last stage indicator isn't used until the dirty ring is enabled in the subsequent patches. No functional change intended. Signed-off-by: Gavin Shan Reviewed-by: Peter Xu Tested-by: Zhenyu Zhang Message-Id: <20230509022122.20888-2-gshan@redhat.com> Signed-off-by: Paolo Bonzini --- accel/kvm/kvm-all.c | 2 +- include/exec/memory.h | 9 +++++++-- migration/dirtyrate.c | 4 ++-- migration/ram.c | 20 ++++++++++---------- softmmu/memory.c | 10 +++++----- 5 files changed, 25 insertions(+), 20 deletions(-) diff --git a/accel/kvm/kvm-all.c b/accel/kvm/kvm-all.c index cf3a88d90e92..870abad82681 100644 --- a/accel/kvm/kvm-all.c +++ b/accel/kvm/kvm-all.c @@ -1563,7 +1563,7 @@ static void kvm_log_sync(MemoryListener *listener, kvm_slots_unlock(); } -static void kvm_log_sync_global(MemoryListener *l) +static void kvm_log_sync_global(MemoryListener *l, bool last_stage) { KVMMemoryListener *kml = container_of(l, KVMMemoryListener, listener); KVMState *s = kvm_state; diff --git a/include/exec/memory.h b/include/exec/memory.h index e45ce6061fb5..6f24a3dd4632 100644 --- a/include/exec/memory.h +++ b/include/exec/memory.h @@ -934,8 +934,11 @@ struct MemoryListener { * its @log_sync must be NULL. Vice versa. * * @listener: The #MemoryListener. + * @last_stage: The last stage to synchronize the log during migration. + * The caller should gurantee that the synchronization with true for + * @last_stage is triggered for once after all VCPUs have been stopped. */ - void (*log_sync_global)(MemoryListener *listener); + void (*log_sync_global)(MemoryListener *listener, bool last_stage); /** * @log_clear: @@ -2422,8 +2425,10 @@ MemoryRegionSection memory_region_find(MemoryRegion *mr, * memory_global_dirty_log_sync: synchronize the dirty log for all memory * * Synchronizes the dirty page log for all address spaces. + * + * @last_stage: whether this is the last stage of live migration */ -void memory_global_dirty_log_sync(void); +void memory_global_dirty_log_sync(bool last_stage); /** * memory_global_dirty_log_sync: synchronize the dirty log for all memory diff --git a/migration/dirtyrate.c b/migration/dirtyrate.c index c06f12c39ded..76257a14918e 100644 --- a/migration/dirtyrate.c +++ b/migration/dirtyrate.c @@ -100,7 +100,7 @@ void global_dirty_log_change(unsigned int flag, bool start) static void global_dirty_log_sync(unsigned int flag, bool one_shot) { qemu_mutex_lock_iothread(); - memory_global_dirty_log_sync(); + memory_global_dirty_log_sync(false); if (one_shot) { memory_global_dirty_log_stop(flag); } @@ -580,7 +580,7 @@ static void calculate_dirtyrate_dirty_bitmap(struct DirtyRateConfig config) * skip it unconditionally and start dirty tracking * from 2'round of log sync */ - memory_global_dirty_log_sync(); + memory_global_dirty_log_sync(false); /* * reset page protect manually and unconditionally. diff --git a/migration/ram.c b/migration/ram.c index f69d8d42b03e..5900cabf2daa 100644 --- a/migration/ram.c +++ b/migration/ram.c @@ -1039,7 +1039,7 @@ static void migration_trigger_throttle(RAMState *rs) } } -static void migration_bitmap_sync(RAMState *rs) +static void migration_bitmap_sync(RAMState *rs, bool last_stage) { RAMBlock *block; int64_t end_time; @@ -1051,7 +1051,7 @@ static void migration_bitmap_sync(RAMState *rs) } trace_migration_bitmap_sync_start(); - memory_global_dirty_log_sync(); + memory_global_dirty_log_sync(last_stage); qemu_mutex_lock(&rs->bitmap_mutex); WITH_RCU_READ_LOCK_GUARD() { @@ -1086,7 +1086,7 @@ static void migration_bitmap_sync(RAMState *rs) } } -static void migration_bitmap_sync_precopy(RAMState *rs) +static void migration_bitmap_sync_precopy(RAMState *rs, bool last_stage) { Error *local_err = NULL; @@ -1099,7 +1099,7 @@ static void migration_bitmap_sync_precopy(RAMState *rs) local_err = NULL; } - migration_bitmap_sync(rs); + migration_bitmap_sync(rs, last_stage); if (precopy_notify(PRECOPY_NOTIFY_AFTER_BITMAP_SYNC, &local_err)) { error_report_err(local_err); @@ -2699,7 +2699,7 @@ void ram_postcopy_send_discard_bitmap(MigrationState *ms) RCU_READ_LOCK_GUARD(); /* This should be our last sync, the src is now paused */ - migration_bitmap_sync(rs); + migration_bitmap_sync(rs, false); /* Easiest way to make sure we don't resume in the middle of a host-page */ rs->pss[RAM_CHANNEL_PRECOPY].last_sent_block = NULL; @@ -2890,7 +2890,7 @@ static void ram_init_bitmaps(RAMState *rs) /* We don't use dirty log with background snapshots */ if (!migrate_background_snapshot()) { memory_global_dirty_log_start(GLOBAL_DIRTY_MIGRATION); - migration_bitmap_sync_precopy(rs); + migration_bitmap_sync_precopy(rs, false); } } qemu_mutex_unlock_ramlist(); @@ -3214,7 +3214,7 @@ static int ram_save_complete(QEMUFile *f, void *opaque) WITH_RCU_READ_LOCK_GUARD() { if (!migration_in_postcopy()) { - migration_bitmap_sync_precopy(rs); + migration_bitmap_sync_precopy(rs, true); } ram_control_before_iterate(f, RAM_CONTROL_FINISH); @@ -3288,7 +3288,7 @@ static void ram_state_pending_exact(void *opaque, uint64_t *must_precopy, if (!migration_in_postcopy() && remaining_size < s->threshold_size) { qemu_mutex_lock_iothread(); WITH_RCU_READ_LOCK_GUARD() { - migration_bitmap_sync_precopy(rs); + migration_bitmap_sync_precopy(rs, false); } qemu_mutex_unlock_iothread(); remaining_size = rs->migration_dirty_pages * TARGET_PAGE_SIZE; @@ -3523,7 +3523,7 @@ void colo_incoming_start_dirty_log(void) qemu_mutex_lock_iothread(); qemu_mutex_lock_ramlist(); - memory_global_dirty_log_sync(); + memory_global_dirty_log_sync(false); WITH_RCU_READ_LOCK_GUARD() { RAMBLOCK_FOREACH_NOT_IGNORED(block) { ramblock_sync_dirty_bitmap(ram_state, block); @@ -3813,7 +3813,7 @@ void colo_flush_ram_cache(void) void *src_host; unsigned long offset = 0; - memory_global_dirty_log_sync(); + memory_global_dirty_log_sync(false); qemu_mutex_lock(&ram_state->bitmap_mutex); WITH_RCU_READ_LOCK_GUARD() { RAMBLOCK_FOREACH_NOT_IGNORED(block) { diff --git a/softmmu/memory.c b/softmmu/memory.c index b7b3386e9d48..342c12151450 100644 --- a/softmmu/memory.c +++ b/softmmu/memory.c @@ -2253,7 +2253,7 @@ void memory_region_set_dirty(MemoryRegion *mr, hwaddr addr, * If memory region `mr' is NULL, do global sync. Otherwise, sync * dirty bitmap for the specified memory region. */ -static void memory_region_sync_dirty_bitmap(MemoryRegion *mr) +static void memory_region_sync_dirty_bitmap(MemoryRegion *mr, bool last_stage) { MemoryListener *listener; AddressSpace *as; @@ -2283,7 +2283,7 @@ static void memory_region_sync_dirty_bitmap(MemoryRegion *mr) * is to do a global sync, because we are not capable to * sync in a finer granularity. */ - listener->log_sync_global(listener); + listener->log_sync_global(listener, last_stage); trace_memory_region_sync_dirty(mr ? mr->name : "(all)", listener->name, 1); } } @@ -2347,7 +2347,7 @@ DirtyBitmapSnapshot *memory_region_snapshot_and_clear_dirty(MemoryRegion *mr, { DirtyBitmapSnapshot *snapshot; assert(mr->ram_block); - memory_region_sync_dirty_bitmap(mr); + memory_region_sync_dirty_bitmap(mr, false); snapshot = cpu_physical_memory_snapshot_and_clear_dirty(mr, addr, size, client); memory_global_after_dirty_log_sync(); return snapshot; @@ -2873,9 +2873,9 @@ bool memory_region_present(MemoryRegion *container, hwaddr addr) return mr && mr != container; } -void memory_global_dirty_log_sync(void) +void memory_global_dirty_log_sync(bool last_stage) { - memory_region_sync_dirty_bitmap(NULL); + memory_region_sync_dirty_bitmap(NULL, last_stage); } void memory_global_after_dirty_log_sync(void) From patchwork Wed May 17 17:44:23 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Paolo Bonzini X-Patchwork-Id: 1782836 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=nongnu.org (client-ip=209.51.188.17; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=) Authentication-Results: legolas.ozlabs.org; dkim=pass (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=OsqZHbP6; dkim-atps=neutral Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-ECDSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4QM0tX2nFTz20dn for ; Thu, 18 May 2023 03:47:32 +1000 (AEST) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1pzLDo-0007Eb-5j; Wed, 17 May 2023 13:45:56 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pzLDi-00076J-ML for qemu-devel@nongnu.org; Wed, 17 May 2023 13:45:51 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pzLDd-00043E-H6 for qemu-devel@nongnu.org; Wed, 17 May 2023 13:45:48 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1684345544; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=07U1cadvQKbzfwknVmjqtMv4mft3/FrIMkIP/Q4GVbo=; b=OsqZHbP6j1baL25V72UodafKsA14iSHxWD0s+tQB7to5xLOVcKFPMgMIwAkmJJ37jnK3T+ uDxtsnrwILRmNU8LxpZ/DbbaG6JF/ckC/IHYeKoooDCMuLoB+hCs/J4IghrF+r4Ugm7L3Y pUNE3Ei+2ozh6TPABHSEKzuXNhADAjo= Received: from mail-wm1-f71.google.com (mail-wm1-f71.google.com [209.85.128.71]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-264-KVpDJ_fUMbWXl7RFM3v6tA-1; Wed, 17 May 2023 13:45:43 -0400 X-MC-Unique: KVpDJ_fUMbWXl7RFM3v6tA-1 Received: by mail-wm1-f71.google.com with SMTP id 5b1f17b1804b1-3f453ff4cdfso4725255e9.2 for ; Wed, 17 May 2023 10:45:43 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1684345542; x=1686937542; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=07U1cadvQKbzfwknVmjqtMv4mft3/FrIMkIP/Q4GVbo=; b=eCBFsofXiNbxTvCDldW+0PXAjA1ivNmksmf6PZs1bv5pisKXQD+kSYFlkJJnEijtiC RHfPbms2SneHlWGytDYVBvPoKpKYfxr94Tqr1WhZ1/yNOCKfqnzmV1e6l28MgYXdq9bI EKXX9j7GnuVtUy7jcV1KE1qihu0FlBK7HYni8bQE5nQ7T19LcXSfYUigdKKvpY5lHCBU tRXKfmizTJC0d78K3aA2gK/o2ib565dJgPdnU6uleqc/Hdzbs0ilaRh0J5wUpBoVZw0/ 4YbWkuKJHtYt6Zm7a3GtxAhfR3RhKnIakBGA3XfmyX6WjQ/nwvzCSQBYjsZcCj9sMaId LnyQ== X-Gm-Message-State: AC+VfDw+OFZ7d9ctKpm3Ubb/GQfF/ELb8Oxr5toGWFIB0C8omtWdgdgw ZLN0S1AUWIXp+h+cHeBWtp/qOX4QvToXUqflhkKYVRp9wOO94a4dmlt9hSG6FJ4DxNpLvL3fk0t kNH5ydWw+7tFsLlY/fxKUfLFxVfuyx59tUq4QkZNOmUIHOvCparMNSUxAsPIMFm+FKwqV5/uYEx o= X-Received: by 2002:a7b:cc15:0:b0:3f4:16bc:bd1b with SMTP id f21-20020a7bcc15000000b003f416bcbd1bmr26302778wmh.39.1684345542033; Wed, 17 May 2023 10:45:42 -0700 (PDT) X-Google-Smtp-Source: ACHHUZ6zE7N0h4V8RuaLsugPrF4qqzvpFWGIMW5yvCWoNJL+zgYZ3wxYtuAeMg2dgQVOvcdsjQmY8Q== X-Received: by 2002:a7b:cc15:0:b0:3f4:16bc:bd1b with SMTP id f21-20020a7bcc15000000b003f416bcbd1bmr26302761wmh.39.1684345541632; Wed, 17 May 2023 10:45:41 -0700 (PDT) Received: from [192.168.10.118] ([2001:b07:6468:f312:9af8:e5f5:7516:fa89]) by smtp.gmail.com with ESMTPSA id g17-20020a05600c311100b003f4283f5c1bsm5178025wmo.2.2023.05.17.10.45.40 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 17 May 2023 10:45:41 -0700 (PDT) From: Paolo Bonzini To: qemu-devel@nongnu.org Cc: Gavin Shan , Peter Xu , Zhenyu Zhang Subject: [PULL 11/68] kvm: Synchronize the backup bitmap in the last stage Date: Wed, 17 May 2023 19:44:23 +0200 Message-Id: <20230517174520.887405-12-pbonzini@redhat.com> X-Mailer: git-send-email 2.40.1 In-Reply-To: <20230517174520.887405-1-pbonzini@redhat.com> References: <20230517174520.887405-1-pbonzini@redhat.com> MIME-Version: 1.0 Received-SPF: pass client-ip=170.10.133.124; envelope-from=pbonzini@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org From: Gavin Shan In the last stage of live migration or memory slot removal, the backup bitmap needs to be synchronized when it has been enabled. Signed-off-by: Gavin Shan Reviewed-by: Peter Xu Tested-by: Zhenyu Zhang Message-Id: <20230509022122.20888-3-gshan@redhat.com> Signed-off-by: Paolo Bonzini --- accel/kvm/kvm-all.c | 11 +++++++++++ include/sysemu/kvm_int.h | 1 + 2 files changed, 12 insertions(+) diff --git a/accel/kvm/kvm-all.c b/accel/kvm/kvm-all.c index 870abad82681..c3aaabf304d7 100644 --- a/accel/kvm/kvm-all.c +++ b/accel/kvm/kvm-all.c @@ -1361,6 +1361,10 @@ static void kvm_set_phys_mem(KVMMemoryListener *kml, */ if (kvm_state->kvm_dirty_ring_size) { kvm_dirty_ring_reap_locked(kvm_state, NULL); + if (kvm_state->kvm_dirty_ring_with_bitmap) { + kvm_slot_sync_dirty_pages(mem); + kvm_slot_get_dirty_log(kvm_state, mem); + } } else { kvm_slot_get_dirty_log(kvm_state, mem); } @@ -1582,6 +1586,12 @@ static void kvm_log_sync_global(MemoryListener *l, bool last_stage) mem = &kml->slots[i]; if (mem->memory_size && mem->flags & KVM_MEM_LOG_DIRTY_PAGES) { kvm_slot_sync_dirty_pages(mem); + + if (s->kvm_dirty_ring_with_bitmap && last_stage && + kvm_slot_get_dirty_log(s, mem)) { + kvm_slot_sync_dirty_pages(mem); + } + /* * This is not needed by KVM_GET_DIRTY_LOG because the * ioctl will unconditionally overwrite the whole region. @@ -3710,6 +3720,7 @@ static void kvm_accel_instance_init(Object *obj) s->kernel_irqchip_split = ON_OFF_AUTO_AUTO; /* KVM dirty ring is by default off */ s->kvm_dirty_ring_size = 0; + s->kvm_dirty_ring_with_bitmap = false; s->notify_vmexit = NOTIFY_VMEXIT_OPTION_RUN; s->notify_window = 0; s->xen_version = 0; diff --git a/include/sysemu/kvm_int.h b/include/sysemu/kvm_int.h index a641c974ea54..511b42bde5c4 100644 --- a/include/sysemu/kvm_int.h +++ b/include/sysemu/kvm_int.h @@ -115,6 +115,7 @@ struct KVMState } *as; uint64_t kvm_dirty_ring_bytes; /* Size of the per-vcpu dirty ring */ uint32_t kvm_dirty_ring_size; /* Number of dirty GFNs per ring */ + bool kvm_dirty_ring_with_bitmap; struct KVMDirtyRingReaper reaper; NotifyVmexitOption notify_vmexit; uint32_t notify_window; From patchwork Wed May 17 17:44:24 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Paolo Bonzini X-Patchwork-Id: 1782877 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=nongnu.org (client-ip=209.51.188.17; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=) Authentication-Results: legolas.ozlabs.org; dkim=pass (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=aXMkclhx; dkim-atps=neutral Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-ECDSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4QM1Bq08drz20dn for ; Thu, 18 May 2023 04:01:39 +1000 (AEST) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1pzLDp-0007GP-80; Wed, 17 May 2023 13:45:57 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pzLDl-0007Ac-RD for qemu-devel@nongnu.org; Wed, 17 May 2023 13:45:55 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pzLDh-00043e-JS for qemu-devel@nongnu.org; Wed, 17 May 2023 13:45:52 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1684345546; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=wMzrIIUF3Z35KfZhCoLyC7JIwz17iRzsNwnLIod0RBw=; b=aXMkclhxod42DEI+MuLBiYoFM5UbsqegF77XHPY30H4KPZwLIEdkYrAYnb8INNV+WY800J tGWCKMSAVbP3+cpO7HYnZyB9YTcZBl7E+I8dwNJnYBteX2heBjVdFuCZ1ePksX2KaOarpr yrBgh96vzuNuelYl3hLRF3om4tyfv5E= Received: from mail-wr1-f72.google.com (mail-wr1-f72.google.com [209.85.221.72]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-308-prQ43NXkO2uvc-zUNOoM7g-1; Wed, 17 May 2023 13:45:45 -0400 X-MC-Unique: prQ43NXkO2uvc-zUNOoM7g-1 Received: by mail-wr1-f72.google.com with SMTP id ffacd0b85a97d-30889491b4aso188242f8f.1 for ; Wed, 17 May 2023 10:45:45 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1684345543; x=1686937543; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=wMzrIIUF3Z35KfZhCoLyC7JIwz17iRzsNwnLIod0RBw=; b=EngO9LkGpZQRxigMUWTHEkzNy38bsZv+kpRtLZ0Ejets20wVDraMF4++6pZ3cnjGDG MIea5djClTnRGlowI3I3fPk1Uw1uFyr8ZGWOwVhxRD94TJJ0KGyfPwEDGylwdpu+N84N p8qa5Q5M1AAl1jfCkcHbLGAMcXOh48HhHuUnxdO8kQ8dx46UCBTvn2U4hq8RPTUFLQLt aybgVFAwR9MjbbRkLjRxmUrxKXvSObeZjg0jdeRAEXUb5D3ZZjl1yR0T5EB/WbcuUaeI dZIppZThS3MgJhYJhzt3LnEoZrYqg+5AC6EEciPzXEIA1avapF2Ss4OgRdMw0T+RNeJi 0n6w== X-Gm-Message-State: AC+VfDwmATonyzCvj2gUXDogDyXSWrdRabG353gvMYnr7zv+rpp/ojDf Yxtwn7Y9+str7uxxuqnBSa78uRFkgCka8IMRF4KE09fLL4gnuu7wh9MVR/CBGp/3OPrkvzML3HE PuYp737wFBzIpmgtFhYEWF66iMV2qPvBtQbbLe/RG5y5fPVAUC5T7Om//Pbmh8MetIBFUHZU3ib I= X-Received: by 2002:adf:fc52:0:b0:2fc:3596:7392 with SMTP id e18-20020adffc52000000b002fc35967392mr1091589wrs.24.1684345543670; Wed, 17 May 2023 10:45:43 -0700 (PDT) X-Google-Smtp-Source: ACHHUZ6sJePfDAAezHaiFoFAvMw7V53hF6MTRDTPcUxIMpXWpHcumGZJrqaLoT5lUWcmDURGPHDH9Q== X-Received: by 2002:adf:fc52:0:b0:2fc:3596:7392 with SMTP id e18-20020adffc52000000b002fc35967392mr1091571wrs.24.1684345543294; Wed, 17 May 2023 10:45:43 -0700 (PDT) Received: from [192.168.10.118] ([2001:b07:6468:f312:9af8:e5f5:7516:fa89]) by smtp.gmail.com with ESMTPSA id j11-20020adff54b000000b003062c609115sm3469172wrp.21.2023.05.17.10.45.42 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 17 May 2023 10:45:42 -0700 (PDT) From: Paolo Bonzini To: qemu-devel@nongnu.org Cc: Gavin Shan , Peter Xu , Zhenyu Zhang Subject: [PULL 12/68] kvm: Add helper kvm_dirty_ring_init() Date: Wed, 17 May 2023 19:44:24 +0200 Message-Id: <20230517174520.887405-13-pbonzini@redhat.com> X-Mailer: git-send-email 2.40.1 In-Reply-To: <20230517174520.887405-1-pbonzini@redhat.com> References: <20230517174520.887405-1-pbonzini@redhat.com> MIME-Version: 1.0 Received-SPF: pass client-ip=170.10.133.124; envelope-from=pbonzini@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org From: Gavin Shan Due to multiple capabilities associated with the dirty ring for different architectures: KVM_CAP_DIRTY_{LOG_RING, LOG_RING_ACQ_REL} for x86 and arm64 separately. There will be more to be done in order to support the dirty ring for arm64. Lets add helper kvm_dirty_ring_init() to enable the dirty ring. With this, the code looks a bit clean. No functional change intended. Signed-off-by: Gavin Shan Reviewed-by: Peter Xu Tested-by: Zhenyu Zhang Message-Id: <20230509022122.20888-4-gshan@redhat.com> Signed-off-by: Paolo Bonzini --- accel/kvm/kvm-all.c | 76 ++++++++++++++++++++++++++++----------------- 1 file changed, 47 insertions(+), 29 deletions(-) diff --git a/accel/kvm/kvm-all.c b/accel/kvm/kvm-all.c index c3aaabf304d7..5d0de9d0a8f8 100644 --- a/accel/kvm/kvm-all.c +++ b/accel/kvm/kvm-all.c @@ -1462,6 +1462,50 @@ static int kvm_dirty_ring_reaper_init(KVMState *s) return 0; } +static int kvm_dirty_ring_init(KVMState *s) +{ + uint32_t ring_size = s->kvm_dirty_ring_size; + uint64_t ring_bytes = ring_size * sizeof(struct kvm_dirty_gfn); + int ret; + + s->kvm_dirty_ring_size = 0; + s->kvm_dirty_ring_bytes = 0; + + /* Bail if the dirty ring size isn't specified */ + if (!ring_size) { + return 0; + } + + /* + * Read the max supported pages. Fall back to dirty logging mode + * if the dirty ring isn't supported. + */ + ret = kvm_vm_check_extension(s, KVM_CAP_DIRTY_LOG_RING); + if (ret <= 0) { + warn_report("KVM dirty ring not available, using bitmap method"); + return 0; + } + + if (ring_bytes > ret) { + error_report("KVM dirty ring size %" PRIu32 " too big " + "(maximum is %ld). Please use a smaller value.", + ring_size, (long)ret / sizeof(struct kvm_dirty_gfn)); + return -EINVAL; + } + + ret = kvm_vm_enable_cap(s, KVM_CAP_DIRTY_LOG_RING, 0, ring_bytes); + if (ret) { + error_report("Enabling of KVM dirty ring failed: %s. " + "Suggested minimum value is 1024.", strerror(-ret)); + return -EIO; + } + + s->kvm_dirty_ring_size = ring_size; + s->kvm_dirty_ring_bytes = ring_bytes; + + return 0; +} + static void kvm_region_add(MemoryListener *listener, MemoryRegionSection *section) { @@ -2531,35 +2575,9 @@ static int kvm_init(MachineState *ms) * Enable KVM dirty ring if supported, otherwise fall back to * dirty logging mode */ - if (s->kvm_dirty_ring_size > 0) { - uint64_t ring_bytes; - - ring_bytes = s->kvm_dirty_ring_size * sizeof(struct kvm_dirty_gfn); - - /* Read the max supported pages */ - ret = kvm_vm_check_extension(s, KVM_CAP_DIRTY_LOG_RING); - if (ret > 0) { - if (ring_bytes > ret) { - error_report("KVM dirty ring size %" PRIu32 " too big " - "(maximum is %ld). Please use a smaller value.", - s->kvm_dirty_ring_size, - (long)ret / sizeof(struct kvm_dirty_gfn)); - ret = -EINVAL; - goto err; - } - - ret = kvm_vm_enable_cap(s, KVM_CAP_DIRTY_LOG_RING, 0, ring_bytes); - if (ret) { - error_report("Enabling of KVM dirty ring failed: %s. " - "Suggested minimum value is 1024.", strerror(-ret)); - goto err; - } - - s->kvm_dirty_ring_bytes = ring_bytes; - } else { - warn_report("KVM dirty ring not available, using bitmap method"); - s->kvm_dirty_ring_size = 0; - } + ret = kvm_dirty_ring_init(s); + if (ret < 0) { + goto err; } /* From patchwork Wed May 17 17:44:25 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Paolo Bonzini X-Patchwork-Id: 1782860 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=nongnu.org (client-ip=209.51.188.17; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=) Authentication-Results: legolas.ozlabs.org; dkim=pass (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=M0ERZkYV; dkim-atps=neutral Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-ECDSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4QM13w2K0Lz20dX for ; Thu, 18 May 2023 03:55:40 +1000 (AEST) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1pzLDq-0007Ia-Di; Wed, 17 May 2023 13:45:58 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pzLDl-0007Ad-RG for qemu-devel@nongnu.org; Wed, 17 May 2023 13:45:55 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.129.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pzLDh-000441-Ru for qemu-devel@nongnu.org; Wed, 17 May 2023 13:45:53 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1684345548; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=YWI/FfXhzmsdFzS5ZamSzT7nRGEgUVsEby4QhBkFkpg=; b=M0ERZkYV8WO2ZqX2b3Y2GRO675RRlYLFs1rLsKu0UMyRW5lhRYEQJXDxFpD1PM14Cn/+cg dHbQYZnadJi6bhgDtt7evFjLE7kltS5ljkdeh+dbIdmXRmnSzfasjFSr/SmKkKkDwZrheG ugY4ll8XIQ8q5tkIJOS1MvAawpROHzk= Received: from mail-wr1-f72.google.com (mail-wr1-f72.google.com [209.85.221.72]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-617-h1Ai2MZ9NGqoHMoJcgFxzQ-1; Wed, 17 May 2023 13:45:46 -0400 X-MC-Unique: h1Ai2MZ9NGqoHMoJcgFxzQ-1 Received: by mail-wr1-f72.google.com with SMTP id ffacd0b85a97d-30635d18e55so432089f8f.2 for ; Wed, 17 May 2023 10:45:46 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1684345545; x=1686937545; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=YWI/FfXhzmsdFzS5ZamSzT7nRGEgUVsEby4QhBkFkpg=; b=kScJ6ga/5j2vtUl79jUyt5qsTUbsZ3JdNPJiGs/WwyEd8qB2isBEOX1EAqkMgdRinK M05O+iZ/PLJPeuwN/ta2SdQ9S71do3meYjKwilN3kVb9D1gOlfI0TEgah9U17Juyp6Q5 YIvNmno7dd/OyI7YFsdnH9zQ/HV1zPv0U8GpGEIhaiGkksw6bwFEQlSaap/BOimFBWcM 5t6yfE9LsbewBA9j2ieb3mv8+lshJu1hAsvLhAkq9I6DP0pMhXZh2SdoqqCkiSFycKKQ oBgZoYhKDcfCNVfNHL4f3sdJgVOVmuQhneXyXMEFn02+DZDlr6f8dnMZ/Q2hPEHSWeRn iyhQ== X-Gm-Message-State: AC+VfDzCtVxZ8d0pS0SLjNfMMN9aC7Jf4PDgeOe1cJIcwwDqxuH7/lw4 fezYLmcfVHnmxcf5k4namy4bWAmrnUgxpW3i6pIsnnZZA03HGp66VrFN+cIoALgwelCRVCip+sD njGLYarIV0yw1OJQPGL89uMk2kSf4dG7dR/M5TQcFh4fnljCC3wkfl3HRSvqEYJz0JtivEXIbYp g= X-Received: by 2002:a5d:63c3:0:b0:306:321c:995b with SMTP id c3-20020a5d63c3000000b00306321c995bmr1389523wrw.41.1684345545246; Wed, 17 May 2023 10:45:45 -0700 (PDT) X-Google-Smtp-Source: ACHHUZ4HzeSCoNPoEzd1Fu6GWtuKruxwlD01zD2hTxhFOyYGHqvz1O2MMF/mSN1tPqi9K2IdPcqOfw== X-Received: by 2002:a5d:63c3:0:b0:306:321c:995b with SMTP id c3-20020a5d63c3000000b00306321c995bmr1389507wrw.41.1684345544879; Wed, 17 May 2023 10:45:44 -0700 (PDT) Received: from [192.168.10.118] ([2001:b07:6468:f312:9af8:e5f5:7516:fa89]) by smtp.gmail.com with ESMTPSA id n3-20020a1c7203000000b003f4285629casm2863819wmc.42.2023.05.17.10.45.43 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 17 May 2023 10:45:44 -0700 (PDT) From: Paolo Bonzini To: qemu-devel@nongnu.org Cc: Gavin Shan , Juan Quintela , Zhenyu Zhang , Peter Xu Subject: [PULL 13/68] kvm: Enable dirty ring for arm64 Date: Wed, 17 May 2023 19:44:25 +0200 Message-Id: <20230517174520.887405-14-pbonzini@redhat.com> X-Mailer: git-send-email 2.40.1 In-Reply-To: <20230517174520.887405-1-pbonzini@redhat.com> References: <20230517174520.887405-1-pbonzini@redhat.com> MIME-Version: 1.0 Received-SPF: pass client-ip=170.10.129.124; envelope-from=pbonzini@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org From: Gavin Shan arm64 has different capability from x86 to enable the dirty ring, which is KVM_CAP_DIRTY_LOG_RING_ACQ_REL. Besides, arm64 also needs the backup bitmap extension (KVM_CAP_DIRTY_LOG_RING_WITH_BITMAP) when 'kvm-arm-gicv3' or 'arm-its-kvm' device is enabled. Here the extension is always enabled and the unnecessary overhead to do the last stage of dirty log synchronization when those two devices aren't used is introduced, but the overhead should be very small and acceptable. The benefit is cover future cases where those two devices are used without modifying the code. Signed-off-by: Gavin Shan Reviewed-by: Juan Quintela Tested-by: Zhenyu Zhang Reviewed-by: Peter Xu Message-Id: <20230509022122.20888-5-gshan@redhat.com> Signed-off-by: Paolo Bonzini --- accel/kvm/kvm-all.c | 23 +++++++++++++++++++++-- 1 file changed, 21 insertions(+), 2 deletions(-) diff --git a/accel/kvm/kvm-all.c b/accel/kvm/kvm-all.c index 5d0de9d0a8f8..7679f397aec0 100644 --- a/accel/kvm/kvm-all.c +++ b/accel/kvm/kvm-all.c @@ -1466,6 +1466,7 @@ static int kvm_dirty_ring_init(KVMState *s) { uint32_t ring_size = s->kvm_dirty_ring_size; uint64_t ring_bytes = ring_size * sizeof(struct kvm_dirty_gfn); + unsigned int capability = KVM_CAP_DIRTY_LOG_RING; int ret; s->kvm_dirty_ring_size = 0; @@ -1480,7 +1481,12 @@ static int kvm_dirty_ring_init(KVMState *s) * Read the max supported pages. Fall back to dirty logging mode * if the dirty ring isn't supported. */ - ret = kvm_vm_check_extension(s, KVM_CAP_DIRTY_LOG_RING); + ret = kvm_vm_check_extension(s, capability); + if (ret <= 0) { + capability = KVM_CAP_DIRTY_LOG_RING_ACQ_REL; + ret = kvm_vm_check_extension(s, capability); + } + if (ret <= 0) { warn_report("KVM dirty ring not available, using bitmap method"); return 0; @@ -1493,13 +1499,26 @@ static int kvm_dirty_ring_init(KVMState *s) return -EINVAL; } - ret = kvm_vm_enable_cap(s, KVM_CAP_DIRTY_LOG_RING, 0, ring_bytes); + ret = kvm_vm_enable_cap(s, capability, 0, ring_bytes); if (ret) { error_report("Enabling of KVM dirty ring failed: %s. " "Suggested minimum value is 1024.", strerror(-ret)); return -EIO; } + /* Enable the backup bitmap if it is supported */ + ret = kvm_vm_check_extension(s, KVM_CAP_DIRTY_LOG_RING_WITH_BITMAP); + if (ret > 0) { + ret = kvm_vm_enable_cap(s, KVM_CAP_DIRTY_LOG_RING_WITH_BITMAP, 0); + if (ret) { + error_report("Enabling of KVM dirty ring's backup bitmap failed: " + "%s. ", strerror(-ret)); + return -EIO; + } + + s->kvm_dirty_ring_with_bitmap = true; + } + s->kvm_dirty_ring_size = ring_size; s->kvm_dirty_ring_bytes = ring_bytes; From patchwork Wed May 17 17:44:26 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Paolo Bonzini X-Patchwork-Id: 1782866 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=nongnu.org (client-ip=209.51.188.17; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=) Authentication-Results: legolas.ozlabs.org; dkim=pass (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=c2MXn5hH; dkim-atps=neutral Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-ECDSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4QM15p0zFHz20dX for ; Thu, 18 May 2023 03:57:18 +1000 (AEST) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1pzLDr-0007KB-Vf; Wed, 17 May 2023 13:46:00 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pzLDo-0007G3-Pe for qemu-devel@nongnu.org; Wed, 17 May 2023 13:45:56 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pzLDl-00044p-KM for qemu-devel@nongnu.org; Wed, 17 May 2023 13:45:56 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1684345550; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=l876oqAb8aNAhYWvZIc/atB5ahSGSdR5tVYDJUVhSc0=; b=c2MXn5hHbI/54Nz4hZSavtGXD7GO0TCcGBq7IcmpYgrrioq7lhLWUP5SD+h4r4AR05mjoB 5Q0bM0BHhGh5Rq6bZBqFJlXbrfXdH93KLWo7nRf7dwdTq3/W4PrW7ZVtciWYLgDCzjiqDh ta1b4TV20UmXGSrbywFJmUYV4Pprfg8= Received: from mail-wm1-f72.google.com (mail-wm1-f72.google.com [209.85.128.72]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-128-FvIONaD8PvyKEtmiqnLpmA-1; Wed, 17 May 2023 13:45:48 -0400 X-MC-Unique: FvIONaD8PvyKEtmiqnLpmA-1 Received: by mail-wm1-f72.google.com with SMTP id 5b1f17b1804b1-3f4fa463282so7685315e9.0 for ; Wed, 17 May 2023 10:45:48 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1684345547; x=1686937547; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=l876oqAb8aNAhYWvZIc/atB5ahSGSdR5tVYDJUVhSc0=; b=CoSjo4TuBd+PbiDmDnTp4PwHvgIlln5LsZ179tN9SjxjhQYfyU5zJ5BTmIz0aPlP4V VN3c1gWuHn/GkE39Bf/xJdA/xJ2pgRYGidCqkpbYyW4eogUtDdk4HtmTHN5B2USOKVTS D7bBz7t9SKyU5/HVdAnfo1wigd68IPJ9+YySkbvjaO3R/yPbLv6UQZWfkDk632Sdru3V Kxin5qNy3EbWqTwxe6mjKjSAwhi8O41wjnTwRWmcfq9qmYQTfB8nT2lSzJ5ieeq1/Ikm sKpyEU96rXmCdo3J1BzFeEcgECY/PUFWf9TeUpYWWWss6WcmeYTNZqMWPXbjhlwmGQpg eYEg== X-Gm-Message-State: AC+VfDys5FNebS3vXpgGdeGBmklJOzLHuBltCDoIvHITHTr3pjABHr+y G+HDQKjGZRpduSsVrTphfU6hzQPV6OKt3KGYk5aCcB/QNr8HWOZyqeLwCwTTz0Bo/OhpqIVUkHT fcMTqBo5hSgykKIec1FKX+hPI9rxXWPtRvxgqV0be2OuPu60GtGZDYFoffTDHTFUlZgjnlAN+Il c= X-Received: by 2002:a05:600c:3658:b0:3f4:2374:3517 with SMTP id y24-20020a05600c365800b003f423743517mr23442540wmq.19.1684345547054; Wed, 17 May 2023 10:45:47 -0700 (PDT) X-Google-Smtp-Source: ACHHUZ6EpzmgVVWnv8LrZ+BwyKt+zx5qo979l/37YQBOXsXEaceFv6kvihlKeZBU3m/HF+dzWKwN7A== X-Received: by 2002:a05:600c:3658:b0:3f4:2374:3517 with SMTP id y24-20020a05600c365800b003f423743517mr23442521wmq.19.1684345546516; Wed, 17 May 2023 10:45:46 -0700 (PDT) Received: from [192.168.10.118] ([2001:b07:6468:f312:9af8:e5f5:7516:fa89]) by smtp.gmail.com with ESMTPSA id c16-20020a7bc850000000b003f42d3111b8sm2894515wml.30.2023.05.17.10.45.45 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 17 May 2023 10:45:46 -0700 (PDT) From: Paolo Bonzini To: qemu-devel@nongnu.org Subject: [PULL 14/68] tcg: round-robin: do not use mb_read for rr_current_cpu Date: Wed, 17 May 2023 19:44:26 +0200 Message-Id: <20230517174520.887405-15-pbonzini@redhat.com> X-Mailer: git-send-email 2.40.1 In-Reply-To: <20230517174520.887405-1-pbonzini@redhat.com> References: <20230517174520.887405-1-pbonzini@redhat.com> MIME-Version: 1.0 Received-SPF: pass client-ip=170.10.133.124; envelope-from=pbonzini@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Signed-off-by: Paolo Bonzini --- accel/tcg/tcg-accel-ops-rr.c | 11 +++++++---- 1 file changed, 7 insertions(+), 4 deletions(-) diff --git a/accel/tcg/tcg-accel-ops-rr.c b/accel/tcg/tcg-accel-ops-rr.c index 5788efa5ff4d..b6d10fa9a250 100644 --- a/accel/tcg/tcg-accel-ops-rr.c +++ b/accel/tcg/tcg-accel-ops-rr.c @@ -72,11 +72,13 @@ static void rr_kick_next_cpu(void) { CPUState *cpu; do { - cpu = qatomic_mb_read(&rr_current_cpu); + cpu = qatomic_read(&rr_current_cpu); if (cpu) { cpu_exit(cpu); } - } while (cpu != qatomic_mb_read(&rr_current_cpu)); + /* Finish kicking this cpu before reading again. */ + smp_mb(); + } while (cpu != qatomic_read(&rr_current_cpu)); } static void rr_kick_thread(void *opaque) @@ -241,8 +243,9 @@ static void *rr_cpu_thread_fn(void *arg) } while (cpu && cpu_work_list_empty(cpu) && !cpu->exit_request) { - + /* Store rr_current_cpu before evaluating cpu_can_run(). */ qatomic_mb_set(&rr_current_cpu, cpu); + current_cpu = cpu; qemu_clock_enable(QEMU_CLOCK_VIRTUAL, @@ -280,7 +283,7 @@ static void *rr_cpu_thread_fn(void *arg) cpu = CPU_NEXT(cpu); } /* while (cpu && !cpu->exit_request).. */ - /* Does not need qatomic_mb_set because a spurious wakeup is okay. */ + /* Does not need a memory barrier because a spurious wakeup is okay. */ qatomic_set(&rr_current_cpu, NULL); if (cpu && cpu->exit_request) { From patchwork Wed May 17 17:44:27 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Paolo Bonzini X-Patchwork-Id: 1782831 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=nongnu.org (client-ip=209.51.188.17; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=) Authentication-Results: legolas.ozlabs.org; dkim=pass (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=G4ASfSl0; dkim-atps=neutral Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-ECDSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4QM0sf3c0Jz20dr for ; Thu, 18 May 2023 03:46:46 +1000 (AEST) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1pzLE9-0007py-Ky; Wed, 17 May 2023 13:46:18 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pzLDt-0007Kv-Jf for qemu-devel@nongnu.org; Wed, 17 May 2023 13:46:05 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pzLDl-00045m-JT for qemu-devel@nongnu.org; Wed, 17 May 2023 13:46:01 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1684345551; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=s4UDkO3Kdy5NCRIXg9Om8yA7mbqwupVdh0F2UuY1x8M=; b=G4ASfSl0uwAj7ZyB1YO3W6HuEkRHPyVBYxAcNbc+voCVIYuHQoPIJ4dGZUvFXJdQlj0luN VWfxUDmDO9ZXf8iOv1AFLRZFMDhwRhBt3J0UbiNHYENooWI9VBrF2UwaNnAsnAcelMcGez U2pslFr8059Yw08nIXVNOVD3pSQQ4Sk= Received: from mail-wr1-f70.google.com (mail-wr1-f70.google.com [209.85.221.70]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-18-KuTnZ1d8NO25jygKnVz8lA-1; Wed, 17 May 2023 13:45:50 -0400 X-MC-Unique: KuTnZ1d8NO25jygKnVz8lA-1 Received: by mail-wr1-f70.google.com with SMTP id ffacd0b85a97d-30479b764f9so451276f8f.0 for ; Wed, 17 May 2023 10:45:50 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1684345548; x=1686937548; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=s4UDkO3Kdy5NCRIXg9Om8yA7mbqwupVdh0F2UuY1x8M=; b=iOx/92xCbxzjl7dPTN8hOvcZv1AzSsLZNLNnfw13pYUue7TX+2xODbCYeEmvoVc5Ce 0uhYfMNe5ZL7fNE0P34VmPuOI/Hmk1L9LXcQvAqNtLbev320RmUsDlu63fbWH14IJgUa xaehioyPO6iWkohcWzSI8w2/G6RdXbLl/tQdwyNUgMbgmBpHOh30uWvO7rdm7rwFTcFt o80LsMRR4/rW7YZsJD2s+V92Cb3BQIP9GYibk9bUlYyYrbyL8N3Mh16U0wwfdGbJVyFq 9X21HS5ZvFhBR3++AWoUpFzTUaj97CuYHf9rAOzq9+/u+qTJjXI54QN6VcnIs/mWjQ33 zh6g== X-Gm-Message-State: AC+VfDzGbK8jmR3igjEyvFW7CGDn8dSAq8Y7G5wao4Aordx9olXxtcor 7wnpWOSwN0McjxXuMv/g//N4oggW/vrYr5Qaj7Ip5JHwIE/avIH2jcqq9mJcEAXXaTzgUgWPh3l 6KxC7MG6qWrKRh2TxBfaIvowrEhh4zkHzeCoG4JN93uQiDECjahNXleMv0pVN6+7tfWU0QkCr/U 4= X-Received: by 2002:adf:db8f:0:b0:306:4125:5f61 with SMTP id u15-20020adfdb8f000000b0030641255f61mr1209058wri.44.1684345548642; Wed, 17 May 2023 10:45:48 -0700 (PDT) X-Google-Smtp-Source: ACHHUZ7Cdjx7f+zrEroDbjYC6CWe3y/K2J2dmlQTjmNZ92jEZWmBuHToPvuIIhdsHZ1g/20azpeZRg== X-Received: by 2002:adf:db8f:0:b0:306:4125:5f61 with SMTP id u15-20020adfdb8f000000b0030641255f61mr1209048wri.44.1684345548137; Wed, 17 May 2023 10:45:48 -0700 (PDT) Received: from [192.168.10.118] ([2001:b07:6468:f312:9af8:e5f5:7516:fa89]) by smtp.gmail.com with ESMTPSA id z26-20020a7bc7da000000b003f50d6ee334sm2831493wmk.47.2023.05.17.10.45.47 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 17 May 2023 10:45:47 -0700 (PDT) From: Paolo Bonzini To: qemu-devel@nongnu.org Subject: [PULL 15/68] coverity: the definitive COMPONENTS.md update Date: Wed, 17 May 2023 19:44:27 +0200 Message-Id: <20230517174520.887405-16-pbonzini@redhat.com> X-Mailer: git-send-email 2.40.1 In-Reply-To: <20230517174520.887405-1-pbonzini@redhat.com> References: <20230517174520.887405-1-pbonzini@redhat.com> MIME-Version: 1.0 Received-SPF: pass client-ip=170.10.133.124; envelope-from=pbonzini@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org The ordering here tries to be logical and matches the one in the website. Signed-off-by: Paolo Bonzini --- scripts/coverity-scan/COMPONENTS.md | 45 +++++++++++++++++++---------- 1 file changed, 30 insertions(+), 15 deletions(-) diff --git a/scripts/coverity-scan/COMPONENTS.md b/scripts/coverity-scan/COMPONENTS.md index 7c48e0f1d215..883da95aff22 100644 --- a/scripts/coverity-scan/COMPONENTS.md +++ b/scripts/coverity-scan/COMPONENTS.md @@ -24,6 +24,9 @@ hppa i386 ~ (/qemu)?((/include)?/hw/i386/.*|/target/i386/.*|/hw/intc/[^/]*apic[^/]*\.c) +loongarch + ~ (/qemu)?((/include)?/hw/(loongarch/.*|.*/loongarch.*)|/target/loongarch/.*) + m68k ~ (/qemu)?((/include)?/hw/m68k/.*|/target/m68k/.*|(/include)?/hw(/.*)?/mcf.*|(/include)?/hw/nubus/.*) @@ -36,11 +39,14 @@ mips nios2 ~ (/qemu)?((/include)?/hw/nios2/.*|/target/nios2/.*) +openrisc + ~ (/qemu)?((/include)?/hw/openrisc/.*|/target/openrisc/.*) + ppc ~ (/qemu)?((/include)?/hw/ppc/.*|/target/ppc/.*|/hw/pci-host/(uninorth.*|dec.*|prep.*|ppc.*)|/hw/misc/macio/.*|(/include)?/hw/.*/(xics|openpic|spapr).*) riscv - ~ (/qemu)?((/include)?/hw/riscv/.*|/target/riscv/.*) + ~ (/qemu)?((/include)?/hw/riscv/.*|/target/riscv/.*|/hw/.*/(riscv_|ibex_|sifive_).*) rx ~ (/qemu)?((/include)?/hw/rx/.*|/target/rx/.*) @@ -54,12 +60,12 @@ sh4 sparc ~ (/qemu)?((/include)?/hw/sparc(64)?.*|/target/sparc/.*|/hw/.*/grlib.*|/hw/display/cg3.c) -tilegx - ~ (/qemu)?(/target/tilegx/.*) - tricore ~ (/qemu)?((/include)?/hw/tricore/.*|/target/tricore/.*) +xtensa + ~ (/qemu)?((/include)?/hw/xtensa/.*|/target/xtensa/.*) + 9pfs ~ (/qemu)?(/hw/9pfs/.*|/fsdev/.*) @@ -73,7 +79,7 @@ char ~ (/qemu)?(/qemu-char\.c|/include/sysemu/char\.h|(/include)?/hw/char/.*) crypto - ~ (/qemu)?((/include)?/crypto/.*|/hw/.*/crypto.*) + ~ (/qemu)?((/include)?/crypto/.*|/hw/.*/.*crypto.*|(/include/sysemu|/backends)/cryptodev.*) disas ~ (/qemu)?((/include)?/disas.*) @@ -100,7 +106,7 @@ net ~ (/qemu)?((/include)?(/hw)?/(net|rdma)/.*) pci - ~ (/qemu)?(/hw/pci.*|/include/hw/pci.*) + ~ (/qemu)?(/include)?/hw/(cxl/|pci).* qemu-ga ~ (/qemu)?(/qga/.*) @@ -108,9 +114,6 @@ qemu-ga scsi ~ (/qemu)?(/scsi/.*|/hw/scsi/.*|/include/hw/scsi/.*) -tcg - ~ (/qemu)?(/accel/tcg/.*|/replay/.*|/(.*/)?softmmu.*) - trace ~ (/qemu)?(/.*trace.*\.[ch]) @@ -126,9 +129,27 @@ user util ~ (/qemu)?(/util/.*|/include/qemu/.*) +vfio + ~ (/qemu)?(/include)?/hw/vfio/.* + +virtio + ~ (/qemu)?(/include)?/hw/virtio/.* + xen ~ (/qemu)?(.*/xen.*) +hvf + ~ (/qemu)?(.*/hvf.*) + +kvm + ~ (/qemu)?(.*/kvm.*) + +tcg + ~ (/qemu)?(/accel/tcg|/replay|/tcg)/.* + +sysemu + ~ (/qemu)?(/softmmu/.*|/accel/.*) + (headers) ~ (/qemu)?(/include/.*) @@ -137,9 +158,3 @@ testlibs tests ~ (/qemu)?(/tests/.*) - -loongarch - ~ (/qemu)?((/include)?/hw/(loongarch/.*|.*/loongarch.*)|/target/loongarch/.*) - -riscv - ~ (/qemu)?((/include)?/hw/riscv/.*|/target/riscv/.*|/hw/.*/(riscv_|ibex_|sifive_).*) From patchwork Wed May 17 17:44:28 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Paolo Bonzini X-Patchwork-Id: 1782879 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=nongnu.org (client-ip=209.51.188.17; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=) Authentication-Results: legolas.ozlabs.org; dkim=pass (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=SyCGU3Hv; dkim-atps=neutral Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-ECDSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4QM1Cn3Rprz20dn for ; Thu, 18 May 2023 04:02:29 +1000 (AEST) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1pzLDr-0007K9-Ta; Wed, 17 May 2023 13:45:59 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pzLDp-0007I4-Vf for qemu-devel@nongnu.org; Wed, 17 May 2023 13:45:58 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.129.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pzLDm-00046x-Kv for qemu-devel@nongnu.org; Wed, 17 May 2023 13:45:57 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1684345552; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=18IKHxyOv9PYot11dos7srKdbqWwToWb1XTwOxkQT8g=; b=SyCGU3HvD8gutuR2crbTifM8yAVbpcnR3YmUcpgzuUzMTafr5mw+E20EGCDloAgj0q3G8c grdTCc2X90RhIRh6C1X78aFZMIN8st5YzgRGrSo6nQUZvqms7TcMB2Nh/h/wdYfKXTppiM wO5UhK0zGU5CG3XeDMoqbMY25qhU8ic= Received: from mail-wr1-f71.google.com (mail-wr1-f71.google.com [209.85.221.71]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-343-rUTxLOMSPfqNEiOXfzub9g-1; Wed, 17 May 2023 13:45:51 -0400 X-MC-Unique: rUTxLOMSPfqNEiOXfzub9g-1 Received: by mail-wr1-f71.google.com with SMTP id ffacd0b85a97d-306281812d6so467173f8f.2 for ; Wed, 17 May 2023 10:45:51 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1684345550; x=1686937550; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=18IKHxyOv9PYot11dos7srKdbqWwToWb1XTwOxkQT8g=; b=NbEkEMltL0CAET0RkgIC+Hy5li1QHEZBIvtbYjAeOB/vB9kxNVYvlDjm70IvQS//Mj NbUmryxpZNzDECf1IsiSAcen1VXXXGN+QEVQ3wsO0SSrgjFYqcUGEP7a4QMrTvJhpx8+ XV9B+hafwCMPwhb8Gdyc+UYIWSy+yINjLvX6IIwTAIDqKTmlJwZemfyAybfjqH7baYrH qFOU2z3Iryb8P+P8zmywBt4vxahPthFjXzm5AnDD5WDyEnHqdbaDFFIE4BsfBX8GFh/f bFoOdNIXR7xbn/xg5/E2GrMpR0bI2SBHBlF51Sel5MlEmqUxWk650mUJj8mX+fosZcW0 +I1g== X-Gm-Message-State: AC+VfDzYkNt34R1Na5H49aUZopfWIa4ONGkaOy5hkbr+SQnfZDiczJK6 +N+JgjF/DKBQBHpXYgF0o+sGJ8jaU1a13Iv4J9qzbLdnB8rOPf90wG0tEzVQyJjxZdO2ZpH+C28 xI0boVUf3JEfshfpccXRJrJ9CFelhNHDTtWQJZPo0xn7jrmOyIaNKjJtWVUyNAFmP9Qi/3Yj7eV I= X-Received: by 2002:adf:f950:0:b0:309:22ba:dcdc with SMTP id q16-20020adff950000000b0030922badcdcmr1479072wrr.42.1684345549915; Wed, 17 May 2023 10:45:49 -0700 (PDT) X-Google-Smtp-Source: ACHHUZ5ykN0hIn3upuUN45IinZOwFfVmOnKKEoNY8qS6xSDtEPZWezBKJ/T1/MRu9Mn3zOoppg1+xw== X-Received: by 2002:adf:f950:0:b0:309:22ba:dcdc with SMTP id q16-20020adff950000000b0030922badcdcmr1479057wrr.42.1684345549647; Wed, 17 May 2023 10:45:49 -0700 (PDT) Received: from [192.168.10.118] ([2001:b07:6468:f312:9af8:e5f5:7516:fa89]) by smtp.gmail.com with ESMTPSA id p4-20020a05600c358400b003f1738d0d13sm3961048wmq.1.2023.05.17.10.45.48 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 17 May 2023 10:45:49 -0700 (PDT) From: Paolo Bonzini To: qemu-devel@nongnu.org Cc: =?utf-8?q?Th=C3=A9o_Maillart?= , qemu-stable@nongnu.org Subject: [PULL 16/68] scsi-generic: fix buffer overflow on block limits inquiry Date: Wed, 17 May 2023 19:44:28 +0200 Message-Id: <20230517174520.887405-17-pbonzini@redhat.com> X-Mailer: git-send-email 2.40.1 In-Reply-To: <20230517174520.887405-1-pbonzini@redhat.com> References: <20230517174520.887405-1-pbonzini@redhat.com> MIME-Version: 1.0 Received-SPF: pass client-ip=170.10.129.124; envelope-from=pbonzini@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=unavailable autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Using linux 6.x guest, at boot time, an inquiry on a scsi-generic device makes qemu crash. This is caused by a buffer overflow when scsi-generic patches the block limits VPD page. Do the operations on a temporary on-stack buffer that is guaranteed to be large enough. Reported-by: ThĂ©o Maillart Analyzed-by: ThĂ©o Maillart Cc: qemu-stable@nongnu.org Signed-off-by: Paolo Bonzini --- hw/scsi/scsi-generic.c | 14 +++++++++----- 1 file changed, 9 insertions(+), 5 deletions(-) diff --git a/hw/scsi/scsi-generic.c b/hw/scsi/scsi-generic.c index ac9fa662b4e3..2417f0ad8479 100644 --- a/hw/scsi/scsi-generic.c +++ b/hw/scsi/scsi-generic.c @@ -191,12 +191,16 @@ static int scsi_handle_inquiry_reply(SCSIGenericReq *r, SCSIDevice *s, int len) if ((s->type == TYPE_DISK || s->type == TYPE_ZBC) && (r->req.cmd.buf[1] & 0x01)) { page = r->req.cmd.buf[2]; - if (page == 0xb0) { + if (page == 0xb0 && r->buflen >= 8) { + uint8_t buf[16] = {}; + uint8_t buf_used = MIN(r->buflen, 16); uint64_t max_transfer = calculate_max_transfer(s); - stl_be_p(&r->buf[8], max_transfer); - /* Also take care of the opt xfer len. */ - stl_be_p(&r->buf[12], - MIN_NON_ZERO(max_transfer, ldl_be_p(&r->buf[12]))); + + memcpy(buf, r->buf, buf_used); + stl_be_p(&buf[8], max_transfer); + stl_be_p(&buf[12], MIN_NON_ZERO(max_transfer, ldl_be_p(&buf[12]))); + memcpy(r->buf + 8, buf + 8, buf_used - 8); + } else if (s->needs_vpd_bl_emulation && page == 0x00 && r->buflen >= 4) { /* * Now we're capable of supplying the VPD Block Limits From patchwork Wed May 17 17:44:29 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Paolo Bonzini X-Patchwork-Id: 1782850 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=nongnu.org (client-ip=209.51.188.17; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=) Authentication-Results: legolas.ozlabs.org; dkim=pass (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=VRHcHQ/k; dkim-atps=neutral Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-ECDSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4QM0zc2Z8bz20KF for ; Thu, 18 May 2023 03:51:56 +1000 (AEST) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1pzLDs-0007KQ-1z; Wed, 17 May 2023 13:46:00 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pzLDq-0007IZ-50 for qemu-devel@nongnu.org; Wed, 17 May 2023 13:45:58 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.129.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pzLDo-000492-Cz for qemu-devel@nongnu.org; Wed, 17 May 2023 13:45:57 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1684345554; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=Vk6JmC+5kO9bSQsl5rzJJ9aOVpIU11LB+iaULrCQgFM=; b=VRHcHQ/kD+pSwlpAz7cwdMH0KsJnHQj8ws7e245ImJiOq5yKIm6l4FIzhtdBxu5Md4h8jS GZe0fkkUrJ6MqxryCEEXTK6r7vQQY8S/oS644e8BwAZ48YCd2zj0GmgUXEc1WIHExlvRw0 5oLPx+nCzvDlxfhPc1Y0r3byIoDN4bE= Received: from mail-wm1-f70.google.com (mail-wm1-f70.google.com [209.85.128.70]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-663-Ql5AxV9wNVaUdQ4jOMTD7g-1; Wed, 17 May 2023 13:45:53 -0400 X-MC-Unique: Ql5AxV9wNVaUdQ4jOMTD7g-1 Received: by mail-wm1-f70.google.com with SMTP id 5b1f17b1804b1-3f42ceb4a4fso4344765e9.0 for ; Wed, 17 May 2023 10:45:53 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1684345551; x=1686937551; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=Vk6JmC+5kO9bSQsl5rzJJ9aOVpIU11LB+iaULrCQgFM=; b=fHAb1CrPHa0sym2pUqHsYF16c0SF13t59Hi/SnEQJxupm/I8A0SwemOYRK6sEuWBeg N82chXAdEaxTXkQmu9qKpWDym+kW3PeZ3Gl9qJlgDRSF7QAylb29DyVSrw6hnyVz43/a DITs8lxT1sQ4LBU5ovMP3/thum3rYJvaKwICr+m/DNBTeRpxWfl76TZVjdbzL/hjwayb /6kt5CqSvLHchytvGbBhQfNbwNz9GZ1TT3ah0z8FEJe4oMyWp+0EPSqnY5ot88FjEUSe BH4EzqXTYoDiGRk2BlOLT4rrQk4nqO/Sy+z2OcAqRmdyokCn1V1WBbMJm93g5n195zSv iZ0w== X-Gm-Message-State: AC+VfDyXyRFvRp9T9jxZLRCF4Z/AbdmR4K2Pev/4wyb55rxpTQ15dN4f m7W6ktiMWxDAR82IDOBkxmRNec6jy6cs1kL3nYesaWmq2K0uyYfMBOsmsRFARS2WQX/eDZ2orzL gSYCoLZqrMFjvn/QkrV0hNTj2SBgs1w5uLcyyjHNBUjvqjQpWtmbyEAvg+5RSIvwzczdRZJGInO o= X-Received: by 2002:a05:600c:2149:b0:3f5:1240:ace4 with SMTP id v9-20020a05600c214900b003f51240ace4mr5172354wml.25.1684345551533; Wed, 17 May 2023 10:45:51 -0700 (PDT) X-Google-Smtp-Source: ACHHUZ55kTWOozPNAM7sEeUrUl+nKpMfAd4GWdPKoMpEuWGrD0kaY55VfoKhhIHE3xdBWEnwVlUCDQ== X-Received: by 2002:a05:600c:2149:b0:3f5:1240:ace4 with SMTP id v9-20020a05600c214900b003f51240ace4mr5172339wml.25.1684345551222; Wed, 17 May 2023 10:45:51 -0700 (PDT) Received: from [192.168.10.118] ([2001:b07:6468:f312:9af8:e5f5:7516:fa89]) by smtp.gmail.com with ESMTPSA id v18-20020a1cf712000000b003edc4788fa0sm2958218wmh.2.2023.05.17.10.45.50 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 17 May 2023 10:45:50 -0700 (PDT) From: Paolo Bonzini To: qemu-devel@nongnu.org Cc: Steve Sistare Subject: [PULL 17/68] make: clean after distclean deletes source files Date: Wed, 17 May 2023 19:44:29 +0200 Message-Id: <20230517174520.887405-18-pbonzini@redhat.com> X-Mailer: git-send-email 2.40.1 In-Reply-To: <20230517174520.887405-1-pbonzini@redhat.com> References: <20230517174520.887405-1-pbonzini@redhat.com> MIME-Version: 1.0 Received-SPF: pass client-ip=170.10.129.124; envelope-from=pbonzini@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org From: Steve Sistare Run 'make distclean' in a tree, and GNUmakefile is removed. But, GNUmakefile is where we change directory to build. Run 'make distclean' or 'make clean' again, and Makefile applies the clean actions, such as this one, at the top level of the tree. For example, it removes the .d source files in 'meson/test cases/d/*/*.d'. find . \( -name '*.so' -o -name '*.dll' -o \ -name '*.[oda]' -o -name '*.gcno' \) -type f \ ! -path ./roms/edk2/ArmPkg/Library/GccLto/liblto-aarch64.a \ ! -path ./roms/edk2/ArmPkg/Library/GccLto/liblto-arm.a \ -exec rm {} + To fix, remove clean and distclean from UNCHECKED_GOALS, so those targets are "checked", meaning that configure must be run before make. However, the check action does not trigger, because clean does not depend on config-host.mak, so change the action to simply throw an error. Signed-off-by: Steve Sistare Message-Id: <1681909700-94095-1-git-send-email-steven.sistare@oracle.com> Signed-off-by: Paolo Bonzini --- Makefile | 6 ++---- 1 file changed, 2 insertions(+), 4 deletions(-) diff --git a/Makefile b/Makefile index e421f8a1f4f7..3c7d67142f13 100644 --- a/Makefile +++ b/Makefile @@ -26,7 +26,7 @@ quiet-command-run = $(if $(V),,$(if $2,printf " %-7s %s\n" $2 $3 && ))$1 quiet-@ = $(if $(V),,@) quiet-command = $(quiet-@)$(call quiet-command-run,$1,$2,$3) -UNCHECKED_GOALS := %clean TAGS cscope ctags dist \ +UNCHECKED_GOALS := TAGS gtags cscope ctags dist \ help check-help print-% \ docker docker-% vm-help vm-test vm-build-% @@ -176,10 +176,8 @@ plugins: endif # $(CONFIG_PLUGIN) else # config-host.mak does not exist -config-host.mak: ifneq ($(filter-out $(UNCHECKED_GOALS),$(MAKECMDGOALS)),$(if $(MAKECMDGOALS),,fail)) - @echo "Please call configure before running make!" - @exit 1 +$(error Please call configure before running make) endif endif # config-host.mak does not exist From patchwork Wed May 17 17:44:30 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Paolo Bonzini X-Patchwork-Id: 1782880 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=nongnu.org (client-ip=209.51.188.17; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=) Authentication-Results: legolas.ozlabs.org; dkim=pass (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=Me6xWzfb; dkim-atps=neutral Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-ECDSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4QM1Ct38tgz20dn for ; Thu, 18 May 2023 04:02:34 +1000 (AEST) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1pzLE0-0007OH-Ey; Wed, 17 May 2023 13:46:09 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pzLDq-0007JO-RB for qemu-devel@nongnu.org; Wed, 17 May 2023 13:45:58 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pzLDp-0004Aq-AL for qemu-devel@nongnu.org; Wed, 17 May 2023 13:45:58 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1684345556; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=sMI/vELh+V1mQrg7S9f/3I9CjXifMeo4Wxxeeo1ATPw=; b=Me6xWzfbuEhL7QOg+MLq/gv9xjTvIJScKbSjz0scZgh5GAjdAHO6fDxTqJ/Cml3ZRRjYcC /+0yDOynnUDoRNqowb1voo0KzBALBWxt5yQZPW2je2KXhYbp4QasEiCnEOaSjdzJ9bW3SD Vv8hJ7tU5Yk2wT2V59hiy+C0KeQtZ6g= Received: from mail-wm1-f69.google.com (mail-wm1-f69.google.com [209.85.128.69]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-588-r12uwnviNr-sAeb2iHso0A-1; Wed, 17 May 2023 13:45:55 -0400 X-MC-Unique: r12uwnviNr-sAeb2iHso0A-1 Received: by mail-wm1-f69.google.com with SMTP id 5b1f17b1804b1-3f426514683so4393635e9.1 for ; Wed, 17 May 2023 10:45:55 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1684345553; x=1686937553; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=sMI/vELh+V1mQrg7S9f/3I9CjXifMeo4Wxxeeo1ATPw=; b=P4t1wZpoVhIP35bSEkJ7780cefbXy1HLnlk1wOab5g1Y8aU1SZd08ndm5Z0cyuGHmv l2RnCVcVjv94kckbtvnEHOPO02Hxfi1UB+ZggZP6KtdvD7U/PykocCASs9xc3N88xFK6 WusZs+piTnqOfz4xnDpP1VC/oHkDypP4sWiMp9S1x19SBYxZLsRqV8plLjQVvCWF9Yff 4HNBzCTI8BF88ze35oyK27NPWbSrmDE+4Iit7FO26j7+Eps81cHnT/1ZLIOPzIlcQsLf ze4wETOGtt1VKuDmvfFeeqpXwLq6VzR2vChCp1rau2/xkHR/xpDj8ouaMX+uC/rNObDc N3Yw== X-Gm-Message-State: AC+VfDxpPjElR1cneHSEjJxjh07FZ1AYEWrB4wm+GLjMg0Holfay1Asv 2yYGlLIK1iscvBtbAim/pxfyfr/fzBLuO5vEs0vXRxNHEmYsPGZgD49wkPX6teUIYmW1c6ACo+W PWmVaOa26VkQefo+GTA7cTECKg7fnoJdnGhObD725UMAKK8fIZhTEjf3/pz4IK2F+Gy00HNoaI0 0= X-Received: by 2002:a5d:6848:0:b0:306:3b5f:45e2 with SMTP id o8-20020a5d6848000000b003063b5f45e2mr1119153wrw.69.1684345553222; Wed, 17 May 2023 10:45:53 -0700 (PDT) X-Google-Smtp-Source: ACHHUZ5d8KRZbTqKJ7p7EUnEn0tW3z7+8SP1p82ckwulroLbljSJhDGqdQ7QyeLbMpNStRq0xvwzaQ== X-Received: by 2002:a5d:6848:0:b0:306:3b5f:45e2 with SMTP id o8-20020a5d6848000000b003063b5f45e2mr1119145wrw.69.1684345552885; Wed, 17 May 2023 10:45:52 -0700 (PDT) Received: from [192.168.10.118] ([2001:b07:6468:f312:9af8:e5f5:7516:fa89]) by smtp.gmail.com with ESMTPSA id d9-20020a1c7309000000b003f423cc11e4sm2874756wmb.16.2023.05.17.10.45.51 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 17 May 2023 10:45:52 -0700 (PDT) From: Paolo Bonzini To: qemu-devel@nongnu.org Cc: John Snow Subject: [PULL 18/68] python: shut up "pip install" during "make check-minreqs" Date: Wed, 17 May 2023 19:44:30 +0200 Message-Id: <20230517174520.887405-19-pbonzini@redhat.com> X-Mailer: git-send-email 2.40.1 In-Reply-To: <20230517174520.887405-1-pbonzini@redhat.com> References: <20230517174520.887405-1-pbonzini@redhat.com> MIME-Version: 1.0 Received-SPF: pass client-ip=170.10.133.124; envelope-from=pbonzini@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org "make check-minreqs" runs pip without the --disable-pip-version-check option, which causes the obnoxious "A new release of pip available" message. Recent versions of pip also complain that some of the dependencies in our virtual environment rely on "setup.py install" instead of providing a pyproject.toml file; apparently it is deprecated to install them directly from pip instead of letting the "wheel" package take care of them. So, install "wheel" in the virtual environment. Signed-off-by: Paolo Bonzini Signed-off-by: John Snow Message-Id: <20230511035435.734312-2-jsnow@redhat.com> Signed-off-by: Paolo Bonzini --- python/Makefile | 9 ++++++--- 1 file changed, 6 insertions(+), 3 deletions(-) diff --git a/python/Makefile b/python/Makefile index c5bd6ff83ac9..47560657d27e 100644 --- a/python/Makefile +++ b/python/Makefile @@ -54,6 +54,7 @@ pipenv check-pipenv: @echo "pipenv was dropped; try 'make check-minreqs' or 'make min-venv'" @exit 1 +PIP_INSTALL = pip install --disable-pip-version-check .PHONY: min-venv min-venv: $(QEMU_MINVENV_DIR) $(QEMU_MINVENV_DIR)/bin/activate $(QEMU_MINVENV_DIR) $(QEMU_MINVENV_DIR)/bin/activate: setup.cfg tests/minreqs.txt @@ -62,10 +63,12 @@ $(QEMU_MINVENV_DIR) $(QEMU_MINVENV_DIR)/bin/activate: setup.cfg tests/minreqs.tx @( \ echo "ACTIVATE $(QEMU_MINVENV_DIR)"; \ . $(QEMU_MINVENV_DIR)/bin/activate; \ + echo "INSTALL wheel $(QEMU_MINVENV_DIR)"; \ + $(PIP_INSTALL) wheel 1>/dev/null; \ echo "INSTALL -r tests/minreqs.txt $(QEMU_MINVENV_DIR)";\ - pip install -r tests/minreqs.txt 1>/dev/null; \ + $(PIP_INSTALL) -r tests/minreqs.txt 1>/dev/null; \ echo "INSTALL -e qemu $(QEMU_MINVENV_DIR)"; \ - pip install -e . 1>/dev/null; \ + $(PIP_INSTALL) -e . 1>/dev/null; \ ) @touch $(QEMU_MINVENV_DIR) @@ -100,7 +103,7 @@ check-dev: dev-venv .PHONY: develop develop: - pip3 install --disable-pip-version-check -e .[devel] + $(PIP_INSTALL) -e .[devel] .PHONY: check check: From patchwork Wed May 17 17:44:31 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Paolo Bonzini X-Patchwork-Id: 1782834 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=nongnu.org (client-ip=209.51.188.17; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=) Authentication-Results: legolas.ozlabs.org; dkim=pass (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=MKVBbcAZ; dkim-atps=neutral Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-ECDSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4QM0tK3yp2z20dn for ; Thu, 18 May 2023 03:47:21 +1000 (AEST) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1pzLE3-0007VP-C6; Wed, 17 May 2023 13:46:13 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pzLDr-0007Jo-KN for qemu-devel@nongnu.org; Wed, 17 May 2023 13:45:59 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.129.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pzLDq-0004C6-4t for qemu-devel@nongnu.org; Wed, 17 May 2023 13:45:59 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1684345557; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=FayYwThs25Dl8i8SOz3tLnsDO2kYo9i3jD8JBO/RWl4=; b=MKVBbcAZXTsx9A0415mivlg2b1JowosxRDmChXLwrnzcVbo6/xXFR9lMR3jixurnk2We8n RQ+KL3E3y7BqBdWak8rp/OWeoHzmVIWRhXLp1E02ihqoT0/OZ1dgverCHR+Iukli2W78k+ c49UfkR/ebK61sltXmJpEzth1kf3ph0= Received: from mail-wm1-f69.google.com (mail-wm1-f69.google.com [209.85.128.69]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-195-DN3BRwohNNq-jvmNhXWF1A-1; Wed, 17 May 2023 13:45:56 -0400 X-MC-Unique: DN3BRwohNNq-jvmNhXWF1A-1 Received: by mail-wm1-f69.google.com with SMTP id 5b1f17b1804b1-3f41ce0a69fso4759415e9.1 for ; Wed, 17 May 2023 10:45:56 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1684345554; x=1686937554; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=FayYwThs25Dl8i8SOz3tLnsDO2kYo9i3jD8JBO/RWl4=; b=CAQZP4HoPJSpzDMIt4EfwVD7AGCNCvZGPkmDb2k8scXh6ftiG+teXdETWyy4dPStOQ BvqIB9b/uxRp0L9EA9I/DVJAB9FnD9DE5IOQdT9QPpe+FwPLbcBaC5J+fgFyRpCqY9uJ yaOFnLqNLPWvHXzVVPmE/SKIUIjXU8aIJQBJJEBPl3pcZZ1vCaVhqTMaElwIzkJfbDne zzA18yBPs72g3tVJWUDmFBnDdkRrVJ+lfM8EOihCfTPx+xBxy6aTbnXHPZ2r+CVUXgzu oc7IZLaHvkC5wAkUDeA/i2demJTgnI723GQNwtR7JPMSkYVryhkeSVFUjMptEZkkaqSw oMjA== X-Gm-Message-State: AC+VfDwd0vDc+VhasUL4LAYz3NTwqWRAy+nT3l00eiHwKP4ByPEDviOJ Tkl8VCsDkAsJJmyjs1spCtzWUCBL8eUHSM9bHwFHw/zRokiippy0sFZsXdr0sI7huRxS3tg1QGr aG8uHIMtt1cJ1Xx3nxxloau3/WznEPvoPTGDosY63SjnPiIzB+qZhao33qQz/fl+ocGn0QI23KH 0= X-Received: by 2002:a7b:c5d6:0:b0:3f0:b1c9:25d4 with SMTP id n22-20020a7bc5d6000000b003f0b1c925d4mr27646337wmk.21.1684345554710; Wed, 17 May 2023 10:45:54 -0700 (PDT) X-Google-Smtp-Source: ACHHUZ4f23DsU4WYnAYiEiJH/2HrQEGfxsAxP1nFunzxXaBA/7OBUFF95QkowQQgXVMgZ9KsUngIcA== X-Received: by 2002:a7b:c5d6:0:b0:3f0:b1c9:25d4 with SMTP id n22-20020a7bc5d6000000b003f0b1c925d4mr27646324wmk.21.1684345554387; Wed, 17 May 2023 10:45:54 -0700 (PDT) Received: from [192.168.10.118] ([2001:b07:6468:f312:9af8:e5f5:7516:fa89]) by smtp.gmail.com with ESMTPSA id x7-20020a05600c21c700b003f4e3ed98ffsm2903759wmj.35.2023.05.17.10.45.53 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 17 May 2023 10:45:53 -0700 (PDT) From: Paolo Bonzini To: qemu-devel@nongnu.org Cc: John Snow , =?utf-8?q?Daniel_P_=2E_Berrang=C3=A9?= Subject: [PULL 19/68] python: update pylint configuration Date: Wed, 17 May 2023 19:44:31 +0200 Message-Id: <20230517174520.887405-20-pbonzini@redhat.com> X-Mailer: git-send-email 2.40.1 In-Reply-To: <20230517174520.887405-1-pbonzini@redhat.com> References: <20230517174520.887405-1-pbonzini@redhat.com> MIME-Version: 1.0 Received-SPF: pass client-ip=170.10.129.124; envelope-from=pbonzini@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org From: John Snow Pylint 2.17.x decided that SocketAddrT was a bad name for a Type Alias for some reason. Sure, fine, whatever. Signed-off-by: John Snow Reviewed-by: Daniel P. BerrangĂ© Message-Id: <20230511035435.734312-3-jsnow@redhat.com> Signed-off-by: Paolo Bonzini --- python/setup.cfg | 1 + 1 file changed, 1 insertion(+) diff --git a/python/setup.cfg b/python/setup.cfg index 9e923d97628f..3f36502954e4 100644 --- a/python/setup.cfg +++ b/python/setup.cfg @@ -132,6 +132,7 @@ good-names=i, fd, # fd = os.open(...) c, # for c in string: ... T, # for TypeVars. See pylint#3401 + SocketAddrT, # Not sure why this is invalid. [pylint.similarities] # Ignore imports when computing similarities. From patchwork Wed May 17 17:44:32 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Paolo Bonzini X-Patchwork-Id: 1782839 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=nongnu.org (client-ip=209.51.188.17; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=) Authentication-Results: legolas.ozlabs.org; dkim=pass (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=gIZfqml2; dkim-atps=neutral Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-ECDSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4QM0vZ71GSz20dr for ; Thu, 18 May 2023 03:48:26 +1000 (AEST) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1pzLEB-000839-CE; Wed, 17 May 2023 13:46:19 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pzLDy-0007Nu-2u for qemu-devel@nongnu.org; Wed, 17 May 2023 13:46:07 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pzLDv-0004Es-Lp for qemu-devel@nongnu.org; Wed, 17 May 2023 13:46:05 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1684345562; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=qTw3Bd3Jy9NfUA8L6LCAlNhcg4RC5pKPciy2XM891pA=; b=gIZfqml2agt6jquzwL5adVEl+HPGvNTKevqQsBbWIjjRfsF0U8sY08vJbOOfgH9JRURLVH XuYEAR5XGdeUfQ7hYxOooaO188EuO4AVSO3bU9Ul2Am1HaUa8JdVGBj8EeLcL3OaGzrk0N SaUVCD7W3mecnq/ZBJi9QEv8sMvpLwU= Received: from mail-wm1-f69.google.com (mail-wm1-f69.google.com [209.85.128.69]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-170-Aiztuw0RNr-Yfo37WvzrBg-1; Wed, 17 May 2023 13:45:59 -0400 X-MC-Unique: Aiztuw0RNr-Yfo37WvzrBg-1 Received: by mail-wm1-f69.google.com with SMTP id 5b1f17b1804b1-3f42d18a079so6571625e9.2 for ; Wed, 17 May 2023 10:45:58 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1684345556; x=1686937556; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=qTw3Bd3Jy9NfUA8L6LCAlNhcg4RC5pKPciy2XM891pA=; b=BwuVTup7YchSM5qP2/7yQkeFkuTRtTVY2N2v13OSL7GbLgpLeheMC5KsEc/Um/e9N1 7Z9a8JMzKBRkn0+rlkztx9ZGS4DQr10E7Ff0yQW9PSZQyLTVsaWUPSoajLXfCosXwBlq SycjhdOOCSNI7sATVI11brVheC94wIOKxQGMZkFL8Ho/Hw/yMnbNtDUsgHivm5zUUplU Qtv5GLZDYZOFGqPCXJmOXxuKxUwdM/6+s8th8F8H4mUB/oIVgIuClnjUP1bRNctSfNVV /jysbn5pclJHqfpCWPn2yWOLDoF9MfEfBZgyVhLHFGqJb8BmT/UOmbM7Mk7nusxWuhIm U4mA== X-Gm-Message-State: AC+VfDxCe2xhqY2Ir7r1KnmYboeOrq7PWXq6YS8I2FE97/mOAoLLGg3J wpkLADJC/huxrBvpAOOFBB+aqF9UcKYgZxNdIPAtkciMW9TaevKWHDVRVitbZ/8U1m1kObMKfn+ 8gr+ATjlUlbk2/VWvg3klAwG08V34Dx9BvknPtG7bXUfvfv5enhsCMXfjhufWFCOaLbgIeexvAX c= X-Received: by 2002:a1c:6a18:0:b0:3f4:294d:8529 with SMTP id f24-20020a1c6a18000000b003f4294d8529mr25305275wmc.19.1684345556394; Wed, 17 May 2023 10:45:56 -0700 (PDT) X-Google-Smtp-Source: ACHHUZ4X/ezRoI4Xi2oNuYYYHIZ2RUkR+5s91M2xXNRAsqNuUxg/yZXS4xsrv2O46zSgQdPEyXjdHg== X-Received: by 2002:a1c:6a18:0:b0:3f4:294d:8529 with SMTP id f24-20020a1c6a18000000b003f4294d8529mr25305258wmc.19.1684345555957; Wed, 17 May 2023 10:45:55 -0700 (PDT) Received: from [192.168.10.118] ([2001:b07:6468:f312:9af8:e5f5:7516:fa89]) by smtp.gmail.com with ESMTPSA id d15-20020adff2cf000000b002cea9d931e6sm3490223wrp.78.2023.05.17.10.45.54 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 17 May 2023 10:45:55 -0700 (PDT) From: Paolo Bonzini To: qemu-devel@nongnu.org Cc: John Snow Subject: [PULL 20/68] python: add mkvenv.py Date: Wed, 17 May 2023 19:44:32 +0200 Message-Id: <20230517174520.887405-21-pbonzini@redhat.com> X-Mailer: git-send-email 2.40.1 In-Reply-To: <20230517174520.887405-1-pbonzini@redhat.com> References: <20230517174520.887405-1-pbonzini@redhat.com> MIME-Version: 1.0 Received-SPF: pass client-ip=170.10.133.124; envelope-from=pbonzini@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org From: John Snow This script will be responsible for building a lightweight Python virtual environment at configure time. It works with Python 3.6 or newer. It has been designed to: - work *offline*, no PyPI required. - work *quickly*, The fast path is only ~65ms on my machine. - work *robustly*, with multiple fallbacks to keep things working. - work *cooperatively*, using system packages where possible. (You can use your distro's meson, no problem.) Due to its unique position in the build chain, it exists outside of the installable python packages in-tree and *must* be runnable without any third party dependencies. Under normal circumstances, the only dependency required to execute this script is Python 3.6+ itself. The script is *faster* by several seconds when setuptools and pip are installed in the host environment, which is probably the case for a typical multi-purpose developer workstation. In the event that pip/setuptools are missing or not usable, additional dependencies may be required on some distributions which remove certain Python stdlib modules to package them separately: - Debian may require python3-venv to provide "ensurepip" - NetBSD may require py310-expat to provide "pyexpat" * (* Or whichever version is current for NetBSD.) Signed-off-by: Paolo Bonzini Signed-off-by: John Snow Message-Id: <20230511035435.734312-4-jsnow@redhat.com> Signed-off-by: Paolo Bonzini --- python/scripts/mkvenv.py | 247 +++++++++++++++++++++++++++++++++++++++ python/setup.cfg | 9 ++ python/tests/flake8.sh | 1 + python/tests/isort.sh | 1 + python/tests/mypy.sh | 1 + python/tests/pylint.sh | 1 + 6 files changed, 260 insertions(+) create mode 100644 python/scripts/mkvenv.py diff --git a/python/scripts/mkvenv.py b/python/scripts/mkvenv.py new file mode 100644 index 000000000000..a4534e41b52b --- /dev/null +++ b/python/scripts/mkvenv.py @@ -0,0 +1,247 @@ +""" +mkvenv - QEMU pyvenv bootstrapping utility + +usage: mkvenv [-h] command ... + +QEMU pyvenv bootstrapping utility + +options: + -h, --help show this help message and exit + +Commands: + command Description + create create a venv + +-------------------------------------------------- + +usage: mkvenv create [-h] target + +positional arguments: + target Target directory to install virtual environment into. + +options: + -h, --help show this help message and exit + +""" + +# Copyright (C) 2022-2023 Red Hat, Inc. +# +# Authors: +# John Snow +# Paolo Bonzini +# +# This work is licensed under the terms of the GNU GPL, version 2 or +# later. See the COPYING file in the top-level directory. + +import argparse +import logging +import os +from pathlib import Path +import subprocess +import sys +from types import SimpleNamespace +from typing import Any, Optional, Union +import venv + + +# Do not add any mandatory dependencies from outside the stdlib: +# This script *must* be usable standalone! + +DirType = Union[str, bytes, "os.PathLike[str]", "os.PathLike[bytes]"] +logger = logging.getLogger("mkvenv") + + +class Ouch(RuntimeError): + """An Exception class we can't confuse with a builtin.""" + + +class QemuEnvBuilder(venv.EnvBuilder): + """ + An extension of venv.EnvBuilder for building QEMU's configure-time venv. + + As of this commit, it does not yet do anything particularly + different than the standard venv-creation utility. The next several + commits will gradually change that in small commits that highlight + each feature individually. + + Parameters for base class init: + - system_site_packages: bool = False + - clear: bool = False + - symlinks: bool = False + - upgrade: bool = False + - with_pip: bool = False + - prompt: Optional[str] = None + - upgrade_deps: bool = False (Since 3.9) + """ + + def __init__(self, *args: Any, **kwargs: Any) -> None: + logger.debug("QemuEnvBuilder.__init__(...)") + super().__init__(*args, **kwargs) + + # Make the context available post-creation: + self._context: Optional[SimpleNamespace] = None + + def ensure_directories(self, env_dir: DirType) -> SimpleNamespace: + logger.debug("ensure_directories(env_dir=%s)", env_dir) + self._context = super().ensure_directories(env_dir) + return self._context + + def get_value(self, field: str) -> str: + """ + Get a string value from the context namespace after a call to build. + + For valid field names, see: + https://docs.python.org/3/library/venv.html#venv.EnvBuilder.ensure_directories + """ + ret = getattr(self._context, field) + assert isinstance(ret, str) + return ret + + +def make_venv( # pylint: disable=too-many-arguments + env_dir: Union[str, Path], + system_site_packages: bool = False, + clear: bool = True, + symlinks: Optional[bool] = None, + with_pip: bool = True, +) -> None: + """ + Create a venv using `QemuEnvBuilder`. + + This is analogous to the `venv.create` module-level convenience + function that is part of the Python stdblib, except it uses + `QemuEnvBuilder` instead. + + :param env_dir: The directory to create/install to. + :param system_site_packages: + Allow inheriting packages from the system installation. + :param clear: When True, fully remove any prior venv and files. + :param symlinks: + Whether to use symlinks to the target interpreter or not. If + left unspecified, it will use symlinks except on Windows to + match behavior with the "venv" CLI tool. + :param with_pip: + Whether to install "pip" binaries or not. + """ + logger.debug( + "%s: make_venv(env_dir=%s, system_site_packages=%s, " + "clear=%s, symlinks=%s, with_pip=%s)", + __file__, + str(env_dir), + system_site_packages, + clear, + symlinks, + with_pip, + ) + + if symlinks is None: + # Default behavior of standard venv CLI + symlinks = os.name != "nt" + + builder = QemuEnvBuilder( + system_site_packages=system_site_packages, + clear=clear, + symlinks=symlinks, + with_pip=with_pip, + ) + + style = "non-isolated" if builder.system_site_packages else "isolated" + print( + f"mkvenv: Creating {style} virtual environment" + f" at '{str(env_dir)}'", + file=sys.stderr, + ) + + try: + logger.debug("Invoking builder.create()") + try: + builder.create(str(env_dir)) + except SystemExit as exc: + # Some versions of the venv module raise SystemExit; *nasty*! + # We want the exception that prompted it. It might be a subprocess + # error that has output we *really* want to see. + logger.debug("Intercepted SystemExit from EnvBuilder.create()") + raise exc.__cause__ or exc.__context__ or exc + logger.debug("builder.create() finished") + except subprocess.CalledProcessError as exc: + logger.error("mkvenv subprocess failed:") + logger.error("cmd: %s", exc.cmd) + logger.error("returncode: %d", exc.returncode) + + def _stringify(data: Union[str, bytes]) -> str: + if isinstance(data, bytes): + return data.decode() + return data + + lines = [] + if exc.stdout: + lines.append("========== stdout ==========") + lines.append(_stringify(exc.stdout)) + lines.append("============================") + if exc.stderr: + lines.append("========== stderr ==========") + lines.append(_stringify(exc.stderr)) + lines.append("============================") + if lines: + logger.error(os.linesep.join(lines)) + + raise Ouch("VENV creation subprocess failed.") from exc + + # print the python executable to stdout for configure. + print(builder.get_value("env_exe")) + + +def _add_create_subcommand(subparsers: Any) -> None: + subparser = subparsers.add_parser("create", help="create a venv") + subparser.add_argument( + "target", + type=str, + action="store", + help="Target directory to install virtual environment into.", + ) + + +def main() -> int: + """CLI interface to make_qemu_venv. See module docstring.""" + if os.environ.get("DEBUG") or os.environ.get("GITLAB_CI"): + # You're welcome. + logging.basicConfig(level=logging.DEBUG) + elif os.environ.get("V"): + logging.basicConfig(level=logging.INFO) + + parser = argparse.ArgumentParser( + prog="mkvenv", + description="QEMU pyvenv bootstrapping utility", + ) + subparsers = parser.add_subparsers( + title="Commands", + dest="command", + metavar="command", + help="Description", + ) + + _add_create_subcommand(subparsers) + + args = parser.parse_args() + try: + if args.command == "create": + make_venv( + args.target, + system_site_packages=True, + clear=True, + ) + logger.debug("mkvenv.py %s: exiting", args.command) + except Ouch as exc: + print("\n*** Ouch! ***\n", file=sys.stderr) + print(str(exc), "\n\n", file=sys.stderr) + return 1 + except SystemExit: + raise + except: # pylint: disable=bare-except + logger.exception("mkvenv did not complete successfully:") + return 2 + return 0 + + +if __name__ == "__main__": + sys.exit(main()) diff --git a/python/setup.cfg b/python/setup.cfg index 3f36502954e4..5b25f810fa8b 100644 --- a/python/setup.cfg +++ b/python/setup.cfg @@ -103,6 +103,15 @@ ignore_missing_imports = True [mypy-pygments] ignore_missing_imports = True +[mypy-importlib.metadata] +ignore_missing_imports = True + +[mypy-importlib_metadata] +ignore_missing_imports = True + +[mypy-pkg_resources] +ignore_missing_imports = True + [pylint.messages control] # Disable the message, report, category or checker with the given id(s). You # can either give multiple identifiers separated by comma (,) or put this diff --git a/python/tests/flake8.sh b/python/tests/flake8.sh index 1cd7d40fad8b..e0136996453f 100755 --- a/python/tests/flake8.sh +++ b/python/tests/flake8.sh @@ -1,2 +1,3 @@ #!/bin/sh -e python3 -m flake8 qemu/ +python3 -m flake8 scripts/ diff --git a/python/tests/isort.sh b/python/tests/isort.sh index 4480405bfb09..66c2f7df0fdc 100755 --- a/python/tests/isort.sh +++ b/python/tests/isort.sh @@ -1,2 +1,3 @@ #!/bin/sh -e python3 -m isort -c qemu/ +python3 -m isort -c scripts/ diff --git a/python/tests/mypy.sh b/python/tests/mypy.sh index 5f980f563bb7..a33a3f58ab39 100755 --- a/python/tests/mypy.sh +++ b/python/tests/mypy.sh @@ -1,2 +1,3 @@ #!/bin/sh -e python3 -m mypy -p qemu +python3 -m mypy scripts/ diff --git a/python/tests/pylint.sh b/python/tests/pylint.sh index 03d64705a107..2b68da90df76 100755 --- a/python/tests/pylint.sh +++ b/python/tests/pylint.sh @@ -1,3 +1,4 @@ #!/bin/sh -e # See commit message for environment variable explainer. SETUPTOOLS_USE_DISTUTILS=stdlib python3 -m pylint qemu/ +SETUPTOOLS_USE_DISTUTILS=stdlib python3 -m pylint scripts/ From patchwork Wed May 17 17:44:33 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Paolo Bonzini X-Patchwork-Id: 1782851 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=nongnu.org (client-ip=209.51.188.17; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=) Authentication-Results: legolas.ozlabs.org; dkim=pass (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=BLwYithX; dkim-atps=neutral Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-ECDSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4QM1026TMMz20KF for ; Thu, 18 May 2023 03:52:18 +1000 (AEST) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1pzLEX-00007v-JM; Wed, 17 May 2023 13:46:41 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pzLDy-0007Nt-2A for qemu-devel@nongnu.org; Wed, 17 May 2023 13:46:07 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pzLDv-0004Ee-La for qemu-devel@nongnu.org; Wed, 17 May 2023 13:46:04 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1684345562; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=6zd8JySzHufb40u2YdjKPP9uyphRfPUF9dyMvW5WUms=; b=BLwYithXrWfNsTaFRLIWP6ayL0UqnO0JaohMeGtZinx1JTU6lNVlNFB1gzXVoXawb8Ft48 5Q89eRDyqpIAMDPu03wc0vvQ/HkMj36FnoO5l8iXVfU9XEqVCzQ3Jiybxl6gAnh7BCt6ck kow5DpWC0W6RGkEGM3UU/wgeGc57Buc= Received: from mail-wm1-f72.google.com (mail-wm1-f72.google.com [209.85.128.72]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-622-kigC6oc8NomsiYmfOLbwlg-1; Wed, 17 May 2023 13:45:59 -0400 X-MC-Unique: kigC6oc8NomsiYmfOLbwlg-1 Received: by mail-wm1-f72.google.com with SMTP id 5b1f17b1804b1-3f33f8ffa37so4308035e9.2 for ; Wed, 17 May 2023 10:45:59 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1684345558; x=1686937558; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=6zd8JySzHufb40u2YdjKPP9uyphRfPUF9dyMvW5WUms=; b=AMRrHFd7BWN0AzEW7Glo88b5x7JrgdZoYp9DL0mWpPCb+VjPExl/EDa2C05VbDY1yU /8MIIgzavM97m0poBoPrmzOr0LbKdtnIK2xd2zTGURx8nP+HyRV8GboLM5TqOl9Y12pz /6k7dponQ+9iM8MaWqOdj8Hy6RDyvBdXYhAo8PiBoTT3/AiaouJX0OEzPN/F1uuOt0fK eCiLxRwAQKh4sSjyRSU3QKvszxruqz4YMzfSCCZuAEqNrG4X7hQn1SJ9dC/w4PvkoFXF Wpdfcm0EeoLCtZiYwRDLnNBidZVHWkYa8woQB+9hf2ddGD1Hpx3F2ayIsUl+JtBlf4Kn qwuA== X-Gm-Message-State: AC+VfDw9Lkl9+f+xSuUxm/n75jM2OUJYp5Z8/1DK7f03rZD3AS6U+zyq Hivvgpf7+1gIQHZ4dE69yTmtC6144W9lOceoCuA78X8bLSTU4+MAMip6EHHf0PEKdyBQpJFhFlV 00rIAhQKs0H6RU11OQUIFQA6sD2upKFDPRMuyWL8LHbopmrMaoL97ADSdZrEPEz0a1HPCfXmcQM s= X-Received: by 2002:adf:e5c7:0:b0:306:3286:69a2 with SMTP id a7-20020adfe5c7000000b00306328669a2mr1325630wrn.48.1684345558030; Wed, 17 May 2023 10:45:58 -0700 (PDT) X-Google-Smtp-Source: ACHHUZ68GWtjoIlO3YeyV7a0KGitnsTvE/s4k99z4/S7DMwUVgk505fwxfPFAG5KOFxNMEGvvFfbmw== X-Received: by 2002:adf:e5c7:0:b0:306:3286:69a2 with SMTP id a7-20020adfe5c7000000b00306328669a2mr1325616wrn.48.1684345557674; Wed, 17 May 2023 10:45:57 -0700 (PDT) Received: from [192.168.10.118] ([2001:b07:6468:f312:9af8:e5f5:7516:fa89]) by smtp.gmail.com with ESMTPSA id t2-20020a5d5342000000b003063176ef0dsm3409608wrv.97.2023.05.17.10.45.56 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 17 May 2023 10:45:56 -0700 (PDT) From: Paolo Bonzini To: qemu-devel@nongnu.org Cc: John Snow Subject: [PULL 21/68] mkvenv: add better error message for broken or missing ensurepip Date: Wed, 17 May 2023 19:44:33 +0200 Message-Id: <20230517174520.887405-22-pbonzini@redhat.com> X-Mailer: git-send-email 2.40.1 In-Reply-To: <20230517174520.887405-1-pbonzini@redhat.com> References: <20230517174520.887405-1-pbonzini@redhat.com> MIME-Version: 1.0 Received-SPF: pass client-ip=170.10.133.124; envelope-from=pbonzini@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org From: John Snow Debian debundles ensurepip for python; NetBSD debundles pyexpat but ensurepip needs pyexpat. Try our best to offer a helpful error message instead of just failing catastrophically. Signed-off-by: John Snow Message-Id: <20230511035435.734312-5-jsnow@redhat.com> Signed-off-by: Paolo Bonzini --- python/scripts/mkvenv.py | 37 +++++++++++++++++++++++++++++++++++++ 1 file changed, 37 insertions(+) diff --git a/python/scripts/mkvenv.py b/python/scripts/mkvenv.py index a4534e41b52b..3c48f8b9baa6 100644 --- a/python/scripts/mkvenv.py +++ b/python/scripts/mkvenv.py @@ -34,6 +34,7 @@ # later. See the COPYING file in the top-level directory. import argparse +from importlib.util import find_spec import logging import os from pathlib import Path @@ -76,6 +77,10 @@ class QemuEnvBuilder(venv.EnvBuilder): def __init__(self, *args: Any, **kwargs: Any) -> None: logger.debug("QemuEnvBuilder.__init__(...)") + + if kwargs.get("with_pip", False): + check_ensurepip() + super().__init__(*args, **kwargs) # Make the context available post-creation: @@ -98,6 +103,38 @@ def get_value(self, field: str) -> str: return ret +def check_ensurepip() -> None: + """ + Check that we have ensurepip. + + Raise a fatal exception with a helpful hint if it isn't available. + """ + if not find_spec("ensurepip"): + msg = ( + "Python's ensurepip module is not found.\n" + "It's normally part of the Python standard library, " + "maybe your distribution packages it separately?\n" + "Either install ensurepip, or alleviate the need for it in the " + "first place by installing pip and setuptools for " + f"'{sys.executable}'.\n" + "(Hint: Debian puts ensurepip in its python3-venv package.)" + ) + raise Ouch(msg) + + # ensurepip uses pyexpat, which can also go missing on us: + if not find_spec("pyexpat"): + msg = ( + "Python's pyexpat module is not found.\n" + "It's normally part of the Python standard library, " + "maybe your distribution packages it separately?\n" + "Either install pyexpat, or alleviate the need for it in the " + "first place by installing pip and setuptools for " + f"'{sys.executable}'.\n\n" + "(Hint: NetBSD's pkgsrc debundles this to e.g. 'py310-expat'.)" + ) + raise Ouch(msg) + + def make_venv( # pylint: disable=too-many-arguments env_dir: Union[str, Path], system_site_packages: bool = False, From patchwork Wed May 17 17:44:34 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Paolo Bonzini X-Patchwork-Id: 1782893 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=nongnu.org (client-ip=209.51.188.17; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=) Authentication-Results: legolas.ozlabs.org; dkim=pass (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=LaOIfTNN; dkim-atps=neutral Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-ECDSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4QM1GD0TxDz20dn for ; Thu, 18 May 2023 04:04:36 +1000 (AEST) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1pzLEb-0000kX-2W; Wed, 17 May 2023 13:46:45 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pzLDy-0007Nz-UR for qemu-devel@nongnu.org; Wed, 17 May 2023 13:46:07 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.129.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pzLDv-0004Eu-NC for qemu-devel@nongnu.org; Wed, 17 May 2023 13:46:06 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1684345563; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=r32waci+7msiLYjT0xk6Q0YNKNwf8HVaX33ysEpRnWc=; b=LaOIfTNNFuyH6QNpWsNrlPSEZ0Nu7N20wN3iQPFxuFJ5JruvXkRBAqwihh/SotTy74unU2 AaZlN7e1d4lm44RZpAbi3KmSjATVr6b5BBeFR11/LHEF/Sl1RzA9zAcfoKusazYIHX1YM3 +ccc0TgOdfFCmpNWOrIHDvN5lsNzNAs= Received: from mail-wm1-f70.google.com (mail-wm1-f70.google.com [209.85.128.70]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-81-oz0eh5JjO32UosaLeFi0IA-1; Wed, 17 May 2023 13:46:01 -0400 X-MC-Unique: oz0eh5JjO32UosaLeFi0IA-1 Received: by mail-wm1-f70.google.com with SMTP id 5b1f17b1804b1-3f426514683so4393905e9.1 for ; Wed, 17 May 2023 10:46:01 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1684345559; x=1686937559; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=r32waci+7msiLYjT0xk6Q0YNKNwf8HVaX33ysEpRnWc=; b=IkcCJO1JduU0QHbAB8bPuaNI0X4qQV+cVcPQR/OngxkWWjQMZQguyddTm4BLHcPBhF ySXKKcET2ZH60XmUnmT+yuRmTjh4CYqBzeaCkEBS9hA9QWXSfmaXyprxYY0YKNp/3f1i uoOg6IMZTNyYbMS+zG/PfyuCs0MNqHr7VVTGYMRCU9RC+KPJvF1RK+0ynJqekg8aIoT+ V6MQB+JgUExcx3RGRFmogAwy1LUAUpApmPIva1oTW7SXzwnRzN7RvOfcHUXDk6VPki0k XTmy8fRFdG4xnLi5aloCj20oE595+hT0cYw47pygXaCN0mPpCe2ypAsvZiYxCpQbUQ5i SlHg== X-Gm-Message-State: AC+VfDz+9OrX8wHLYJMes5Agr1AhoH0mVakv4bqh5rt6W2ntGGZ4KWPh 7cRbftURdX40ys3vgAtSpoonSprKFxdMg6pRKiNUBS1l8y3D4T3pTg9+C9RXluxuQrK8FVep1v6 X74H935kUw05VpdoDKoKbemLlYaIa4lJhsKTxoguKam0puLlRJTaUojAV2rPq2QHHtRO0UxmFX8 Y= X-Received: by 2002:a5d:464d:0:b0:2f2:783f:ae4a with SMTP id j13-20020a5d464d000000b002f2783fae4amr1358197wrs.32.1684345559678; Wed, 17 May 2023 10:45:59 -0700 (PDT) X-Google-Smtp-Source: ACHHUZ61aEV5iCfVRTSTYTk7OSdZ4OkF4S8/Dzt7rLO5teN5Vt9rWjEl3hAdsJFnB/kPXm45DJlZOQ== X-Received: by 2002:a5d:464d:0:b0:2f2:783f:ae4a with SMTP id j13-20020a5d464d000000b002f2783fae4amr1358189wrs.32.1684345559337; Wed, 17 May 2023 10:45:59 -0700 (PDT) Received: from [192.168.10.118] ([2001:b07:6468:f312:9af8:e5f5:7516:fa89]) by smtp.gmail.com with ESMTPSA id f3-20020a0560001b0300b003063772a55bsm3409470wrz.61.2023.05.17.10.45.58 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 17 May 2023 10:45:58 -0700 (PDT) From: Paolo Bonzini To: qemu-devel@nongnu.org Cc: John Snow Subject: [PULL 22/68] mkvenv: add nested venv workaround Date: Wed, 17 May 2023 19:44:34 +0200 Message-Id: <20230517174520.887405-23-pbonzini@redhat.com> X-Mailer: git-send-email 2.40.1 In-Reply-To: <20230517174520.887405-1-pbonzini@redhat.com> References: <20230517174520.887405-1-pbonzini@redhat.com> MIME-Version: 1.0 Received-SPF: pass client-ip=170.10.129.124; envelope-from=pbonzini@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org From: John Snow Python virtual environments do not typically nest; they may inherit from the top-level system packages or not at all. For our purposes, it would be convenient to emulate "nested" virtual environments to allow callers of the configure script to install specific versions of python utilities in order to test build system features, utility version compatibility, etc. While it is possible to install packages into the system environment (say, by using the --user flag), it's nicer to install test packages into a totally isolated environment instead. As detailed in https://www.qemu.org/2023/03/24/python/, Emulate a nested venv environment by using .pth files installed into the site-packages folder that points to the parent environment when appropriate. Signed-off-by: Paolo Bonzini Signed-off-by: John Snow Message-Id: <20230511035435.734312-6-jsnow@redhat.com> Signed-off-by: Paolo Bonzini --- python/scripts/mkvenv.py | 91 +++++++++++++++++++++++++++++++++++++--- 1 file changed, 86 insertions(+), 5 deletions(-) diff --git a/python/scripts/mkvenv.py b/python/scripts/mkvenv.py index 3c48f8b9baa6..1c1dd89a9afd 100644 --- a/python/scripts/mkvenv.py +++ b/python/scripts/mkvenv.py @@ -38,8 +38,10 @@ import logging import os from pathlib import Path +import site import subprocess import sys +import sysconfig from types import SimpleNamespace from typing import Any, Optional, Union import venv @@ -52,6 +54,11 @@ logger = logging.getLogger("mkvenv") +def inside_a_venv() -> bool: + """Returns True if it is executed inside of a virtual environment.""" + return sys.prefix != sys.base_prefix + + class Ouch(RuntimeError): """An Exception class we can't confuse with a builtin.""" @@ -60,10 +67,9 @@ class QemuEnvBuilder(venv.EnvBuilder): """ An extension of venv.EnvBuilder for building QEMU's configure-time venv. - As of this commit, it does not yet do anything particularly - different than the standard venv-creation utility. The next several - commits will gradually change that in small commits that highlight - each feature individually. + The primary difference is that it emulates a "nested" virtual + environment when invoked from inside of an existing virtual + environment by including packages from the parent. Parameters for base class init: - system_site_packages: bool = False @@ -78,6 +84,18 @@ class QemuEnvBuilder(venv.EnvBuilder): def __init__(self, *args: Any, **kwargs: Any) -> None: logger.debug("QemuEnvBuilder.__init__(...)") + # For nested venv emulation: + self.use_parent_packages = False + if inside_a_venv(): + # Include parent packages only if we're in a venv and + # system_site_packages was True. + self.use_parent_packages = kwargs.pop( + "system_site_packages", False + ) + # Include system_site_packages only when the parent, + # The venv we are currently in, also does so. + kwargs["system_site_packages"] = sys.base_prefix in site.PREFIXES + if kwargs.get("with_pip", False): check_ensurepip() @@ -86,11 +104,71 @@ def __init__(self, *args: Any, **kwargs: Any) -> None: # Make the context available post-creation: self._context: Optional[SimpleNamespace] = None + def get_parent_libpath(self) -> Optional[str]: + """Return the libpath of the parent venv, if applicable.""" + if self.use_parent_packages: + return sysconfig.get_path("purelib") + return None + + @staticmethod + def compute_venv_libpath(context: SimpleNamespace) -> str: + """ + Compatibility wrapper for context.lib_path for Python < 3.12 + """ + # Python 3.12+, not strictly necessary because it's documented + # to be the same as 3.10 code below: + if sys.version_info >= (3, 12): + return context.lib_path + + # Python 3.10+ + if "venv" in sysconfig.get_scheme_names(): + lib_path = sysconfig.get_path( + "purelib", scheme="venv", vars={"base": context.env_dir} + ) + assert lib_path is not None + return lib_path + + # For Python <= 3.9 we need to hardcode this. Fortunately the + # code below was the same in Python 3.6-3.10, so there is only + # one case. + if sys.platform == "win32": + return os.path.join(context.env_dir, "Lib", "site-packages") + return os.path.join( + context.env_dir, + "lib", + "python%d.%d" % sys.version_info[:2], + "site-packages", + ) + def ensure_directories(self, env_dir: DirType) -> SimpleNamespace: logger.debug("ensure_directories(env_dir=%s)", env_dir) self._context = super().ensure_directories(env_dir) return self._context + def create(self, env_dir: DirType) -> None: + logger.debug("create(env_dir=%s)", env_dir) + super().create(env_dir) + assert self._context is not None + self.post_post_setup(self._context) + + def post_post_setup(self, context: SimpleNamespace) -> None: + """ + The final, final hook. Enter the venv and run commands inside of it. + """ + if self.use_parent_packages: + # We're inside of a venv and we want to include the parent + # venv's packages. + parent_libpath = self.get_parent_libpath() + assert parent_libpath is not None + logger.debug("parent_libpath: %s", parent_libpath) + + our_libpath = self.compute_venv_libpath(context) + logger.debug("our_libpath: %s", our_libpath) + + pth_file = os.path.join(our_libpath, "nested.pth") + with open(pth_file, "w", encoding="UTF-8") as file: + file.write(parent_libpath + os.linesep) + def get_value(self, field: str) -> str: """ Get a string value from the context namespace after a call to build. @@ -183,9 +261,12 @@ def make_venv( # pylint: disable=too-many-arguments ) style = "non-isolated" if builder.system_site_packages else "isolated" + nested = "" + if builder.use_parent_packages: + nested = f"(with packages from '{builder.get_parent_libpath()}') " print( f"mkvenv: Creating {style} virtual environment" - f" at '{str(env_dir)}'", + f" {nested}at '{str(env_dir)}'", file=sys.stderr, ) From patchwork Wed May 17 17:44:35 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Paolo Bonzini X-Patchwork-Id: 1782858 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=nongnu.org (client-ip=209.51.188.17; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=) Authentication-Results: legolas.ozlabs.org; dkim=pass (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=f6hyxeoR; dkim-atps=neutral Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-ECDSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4QM13M5G1hz20dX for ; Thu, 18 May 2023 03:55:11 +1000 (AEST) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1pzLED-0008ED-0S; Wed, 17 May 2023 13:46:21 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pzLE0-0007PT-F4 for qemu-devel@nongnu.org; Wed, 17 May 2023 13:46:09 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pzLDy-0004FZ-Ev for qemu-devel@nongnu.org; Wed, 17 May 2023 13:46:08 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1684345565; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=3gHJKq6jYQjX4YhOcV7zRqLE5WmNW1w8pnBwgnMBv+M=; b=f6hyxeoRawX3X3Uh62PrBhUfPQykNoazp8AKdKa+5NAwOEh6zb9zq+9oqGIlxLlszsZ1Wl 40NsHwv4cKxWfcswTY0w8DuvFbmXbYo2k/l0VgHyhfXtffTjNPmrncNzz1VaqPZskaf8kp gt0uC/v83WNiyXGBGBKMe7hHAqaOUKY= Received: from mail-wm1-f72.google.com (mail-wm1-f72.google.com [209.85.128.72]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-600-UNIW7RMiM5G4qfhdsSHY6A-1; Wed, 17 May 2023 13:46:04 -0400 X-MC-Unique: UNIW7RMiM5G4qfhdsSHY6A-1 Received: by mail-wm1-f72.google.com with SMTP id 5b1f17b1804b1-3f42b36733aso4438195e9.3 for ; Wed, 17 May 2023 10:46:03 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1684345561; x=1686937561; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=3gHJKq6jYQjX4YhOcV7zRqLE5WmNW1w8pnBwgnMBv+M=; b=cO8g4mny6Gxw4ZnwyuCxamY0bGH0UP1eKjTGHbFlhtiv9MwfhFeEPJDSMhEAP+0qjJ B4d2Eii+OJIfDc5TD/zmil+UA1MiDKrqNWuRlJxS8ARDbeuV/vTsX6QMyHjrXsX2KhxR nyDzwGT3yBvFktF7HChxXk1G/kWWFrGVPVru3GAs8vpfve3kqL7sZEY6HZliq1C3X3te ay3a4aFOz00UL+0ib4iJieM39oW8eXCpeDjJQ8Hck0fihTnGJjyF6fpOomYXiVMMGDX0 Vv/Jxoi75yzFRg8CHjB0upeCHpZPpJx/6sTATR7wUhDBV2fL/vOKGs4ZmpfH8zWu6q17 7E1g== X-Gm-Message-State: AC+VfDwtTpHvM2zMWHzw2LIl4EyDEGZRnK3/YdhcAAXNd1+16/CwvgWv OC69WrE+0b714Lf4rcxf7PusH6FGFTOl9kceixpuChLnFtJrV2Ak0ZPPJdl0eALO1rG3WO+To8y kC/87KKOjzEacsmb5+YFnHyiielMm0A4i8ElKBOKDkhegAcFXsI5UG9nAnZ8CgaUuWAhAMqW7Fk k= X-Received: by 2002:a05:600c:22ca:b0:3f4:2158:289b with SMTP id 10-20020a05600c22ca00b003f42158289bmr27581622wmg.18.1684345561588; Wed, 17 May 2023 10:46:01 -0700 (PDT) X-Google-Smtp-Source: ACHHUZ4v9WkxxR46+kwPIueTZk7CL+iQ5bPiFAeLmN/jHpdMKpz9pv0bJsXe/6ZcEKF/IRq0x7VLmg== X-Received: by 2002:a05:600c:22ca:b0:3f4:2158:289b with SMTP id 10-20020a05600c22ca00b003f42158289bmr27581604wmg.18.1684345561043; Wed, 17 May 2023 10:46:01 -0700 (PDT) Received: from [192.168.10.118] ([2001:b07:6468:f312:9af8:e5f5:7516:fa89]) by smtp.gmail.com with ESMTPSA id o14-20020a5d58ce000000b0030629536e64sm3384478wrf.30.2023.05.17.10.45.59 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 17 May 2023 10:46:00 -0700 (PDT) From: Paolo Bonzini To: qemu-devel@nongnu.org Cc: John Snow Subject: [PULL 23/68] mkvenv: add ensure subcommand Date: Wed, 17 May 2023 19:44:35 +0200 Message-Id: <20230517174520.887405-24-pbonzini@redhat.com> X-Mailer: git-send-email 2.40.1 In-Reply-To: <20230517174520.887405-1-pbonzini@redhat.com> References: <20230517174520.887405-1-pbonzini@redhat.com> MIME-Version: 1.0 Received-SPF: pass client-ip=170.10.133.124; envelope-from=pbonzini@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org From: John Snow This command is to be used to add various packages (or ensure they're already present) into the configure-provided venv in a modular fashion. Examples: mkvenv ensure --online --dir "${source_dir}/python/wheels/" "meson>=0.61.5" mkvenv ensure --online "sphinx>=1.6.0" mkvenv ensure "qemu.qmp==0.0.2" It's designed to look for packages in three places, in order: (1) In system packages, if the version installed is already good enough. This way your distribution-provided meson, sphinx, etc are always used as first preference. (2) In a vendored packages directory. Here I am suggesting qemu.git/python/wheels/ as that directory. This is intended to serve as a replacement for vendoring the meson source for QEMU tarballs. It is also highly likely to be extremely useful for packaging the "qemu.qmp" package in source distributions for platforms that do not yet package qemu.qmp separately. (3) Online, via PyPI, ***only when "--online" is passed***. This is only ever used as a fallback if the first two sources do not have an appropriate package that meets the requirement. The ability to build QEMU and run tests *completely offline* is not impinged. Signed-off-by: Paolo Bonzini Signed-off-by: John Snow Message-Id: <20230511035435.734312-7-jsnow@redhat.com> [Use distlib to lookup distributions. - Paolo] Signed-off-by: Paolo Bonzini --- python/scripts/mkvenv.py | 135 ++++++++++++++++++++++++++++++++++++++- python/setup.cfg | 10 +++ python/tests/minreqs.txt | 3 + 3 files changed, 145 insertions(+), 3 deletions(-) diff --git a/python/scripts/mkvenv.py b/python/scripts/mkvenv.py index 1c1dd89a9afd..f6c4f37f84dd 100644 --- a/python/scripts/mkvenv.py +++ b/python/scripts/mkvenv.py @@ -11,6 +11,7 @@ Commands: command Description create create a venv + ensure Ensure that the specified package is installed. -------------------------------------------------- @@ -22,6 +23,18 @@ options: -h, --help show this help message and exit +-------------------------------------------------- + +usage: mkvenv ensure [-h] [--online] [--dir DIR] dep_spec... + +positional arguments: + dep_spec PEP 508 Dependency specification, e.g. 'meson>=0.61.5' + +options: + -h, --help show this help message and exit + --online Install packages from PyPI, if necessary. + --dir DIR Path to vendored packages where we may install from. + """ # Copyright (C) 2022-2023 Red Hat, Inc. @@ -43,8 +56,17 @@ import sys import sysconfig from types import SimpleNamespace -from typing import Any, Optional, Union +from typing import ( + Any, + Optional, + Sequence, + Union, +) import venv +import warnings + +import distlib.database +import distlib.version # Do not add any mandatory dependencies from outside the stdlib: @@ -309,6 +331,77 @@ def _stringify(data: Union[str, bytes]) -> str: print(builder.get_value("env_exe")) +def pip_install( + args: Sequence[str], + online: bool = False, + wheels_dir: Optional[Union[str, Path]] = None, +) -> None: + """ + Use pip to install a package or package(s) as specified in @args. + """ + loud = bool( + os.environ.get("DEBUG") + or os.environ.get("GITLAB_CI") + or os.environ.get("V") + ) + + full_args = [ + sys.executable, + "-m", + "pip", + "install", + "--disable-pip-version-check", + "-v" if loud else "-q", + ] + if not online: + full_args += ["--no-index"] + if wheels_dir: + full_args += ["--find-links", f"file://{str(wheels_dir)}"] + full_args += list(args) + subprocess.run( + full_args, + check=True, + ) + + +def ensure( + dep_specs: Sequence[str], + online: bool = False, + wheels_dir: Optional[Union[str, Path]] = None, +) -> None: + """ + Use pip to ensure we have the package specified by @dep_specs. + + If the package is already installed, do nothing. If online and + wheels_dir are both provided, prefer packages found in wheels_dir + first before connecting to PyPI. + + :param dep_specs: + PEP 508 dependency specifications. e.g. ['meson>=0.61.5']. + :param online: If True, fall back to PyPI. + :param wheels_dir: If specified, search this path for packages. + """ + with warnings.catch_warnings(): + warnings.filterwarnings( + "ignore", category=UserWarning, module="distlib" + ) + dist_path = distlib.database.DistributionPath(include_egg=True) + absent = [] + for spec in dep_specs: + matcher = distlib.version.LegacyMatcher(spec) + dist = dist_path.get_distribution(matcher.name) + if dist is None or not matcher.match(dist.version): + absent.append(spec) + else: + logger.info("found %s", dist) + + if absent: + # Some packages are missing or aren't a suitable version, + # install a suitable (possibly vendored) package. + print(f"mkvenv: installing {', '.join(absent)}", file=sys.stderr) + pip_install(args=absent, online=online, wheels_dir=wheels_dir) + + def _add_create_subcommand(subparsers: Any) -> None: subparser = subparsers.add_parser("create", help="create a venv") subparser.add_argument( @@ -319,13 +412,42 @@ def _add_create_subcommand(subparsers: Any) -> None: ) +def _add_ensure_subcommand(subparsers: Any) -> None: + subparser = subparsers.add_parser( + "ensure", help="Ensure that the specified package is installed." + ) + subparser.add_argument( + "--online", + action="store_true", + help="Install packages from PyPI, if necessary.", + ) + subparser.add_argument( + "--dir", + type=str, + action="store", + help="Path to vendored packages where we may install from.", + ) + subparser.add_argument( + "dep_specs", + type=str, + action="store", + help="PEP 508 Dependency specification, e.g. 'meson>=0.61.5'", + nargs="+", + ) + + def main() -> int: """CLI interface to make_qemu_venv. See module docstring.""" if os.environ.get("DEBUG") or os.environ.get("GITLAB_CI"): # You're welcome. logging.basicConfig(level=logging.DEBUG) - elif os.environ.get("V"): - logging.basicConfig(level=logging.INFO) + else: + if os.environ.get("V"): + logging.basicConfig(level=logging.INFO) + + # These are incredibly noisy even for V=1 + logging.getLogger("distlib.metadata").addFilter(lambda record: False) + logging.getLogger("distlib.database").addFilter(lambda record: False) parser = argparse.ArgumentParser( prog="mkvenv", @@ -339,6 +461,7 @@ def main() -> int: ) _add_create_subcommand(subparsers) + _add_ensure_subcommand(subparsers) args = parser.parse_args() try: @@ -348,6 +471,12 @@ def main() -> int: system_site_packages=True, clear=True, ) + if args.command == "ensure": + ensure( + dep_specs=args.dep_specs, + online=args.online, + wheels_dir=args.dir, + ) logger.debug("mkvenv.py %s: exiting", args.command) except Ouch as exc: print("\n*** Ouch! ***\n", file=sys.stderr) diff --git a/python/setup.cfg b/python/setup.cfg index 5b25f810fa8b..d680374b2950 100644 --- a/python/setup.cfg +++ b/python/setup.cfg @@ -36,6 +36,7 @@ packages = # Remember to update tests/minreqs.txt if changing anything below: devel = avocado-framework >= 90.0 + distlib >= 0.3.6 flake8 >= 3.6.0 fusepy >= 2.0.4 isort >= 5.1.2 @@ -112,6 +113,15 @@ ignore_missing_imports = True [mypy-pkg_resources] ignore_missing_imports = True +[mypy-distlib] +ignore_missing_imports = True + +[mypy-distlib.database] +ignore_missing_imports = True + +[mypy-distlib.version] +ignore_missing_imports = True + [pylint.messages control] # Disable the message, report, category or checker with the given id(s). You # can either give multiple identifiers separated by comma (,) or put this diff --git a/python/tests/minreqs.txt b/python/tests/minreqs.txt index dfb8abb155f4..7ecf5e7fe483 100644 --- a/python/tests/minreqs.txt +++ b/python/tests/minreqs.txt @@ -16,6 +16,9 @@ urwid==2.1.2 urwid-readline==0.13 Pygments==2.9.0 +# Dependencies for mkvenv +distlib==0.3.6 + # Dependencies for FUSE support for qom-fuse fusepy==2.0.4 From patchwork Wed May 17 17:44:36 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Paolo Bonzini X-Patchwork-Id: 1782841 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=nongnu.org (client-ip=209.51.188.17; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=) Authentication-Results: legolas.ozlabs.org; dkim=pass (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=KxWZUFFK; dkim-atps=neutral Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-ECDSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4QM0wq04Jsz20dn for ; Thu, 18 May 2023 03:49:30 +1000 (AEST) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1pzLEW-000059-SH; Wed, 17 May 2023 13:46:41 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pzLE2-0007YR-VU for qemu-devel@nongnu.org; Wed, 17 May 2023 13:46:11 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pzLE0-0004GB-70 for qemu-devel@nongnu.org; Wed, 17 May 2023 13:46:10 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1684345567; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=lkzMnyG/yk3jOYlnZDLrqP+HbKCXIYSIMttPCZPZl5g=; b=KxWZUFFKAMbKT/VJQiZMvKq/D6x5XpSYMpSGlsEwl1+R1PlGsCexeAWyWj4qlI3BlU9gyY lp4+ePG3tcq1FQe63Az3mAcSL6SSqSJrgedJWaTwsA9GN8zLF00TONjqynWr4sxjuarqYi S8OaVWVX0JqHI5a8TNkHz4majT1PX5c= Received: from mail-wm1-f71.google.com (mail-wm1-f71.google.com [209.85.128.71]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-664-55jvs1LZMdC-0tWj-u7XNA-1; Wed, 17 May 2023 13:46:05 -0400 X-MC-Unique: 55jvs1LZMdC-0tWj-u7XNA-1 Received: by mail-wm1-f71.google.com with SMTP id 5b1f17b1804b1-3f426ffdbc6so6545865e9.3 for ; Wed, 17 May 2023 10:46:04 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1684345563; x=1686937563; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=lkzMnyG/yk3jOYlnZDLrqP+HbKCXIYSIMttPCZPZl5g=; b=jHF8G9n9qDruXfV28NUxphMmMnE9nDbALJsSLY+9BMttiLTiKJQt4b2hs3Vz1eCAaS e27YHOYl/cOqDc/cXlD4UK8ek3dnOsSs6zFYzRCF3KQ7ymJo485SGp5fZS+xMdePUyi2 QXSe6y1HuD+7zGpoBmT0f4nwjUjpLh+beVwtpNuxThlZU/Q7RU7wPJqKDJRIvWkiHnhR v++8YUuCcCzH44jpiQTr6D4+X4nopFy7lROTZ3EClVeqSRtCmv3AnH5WnL7+iNZG1XtN wgpSMKnSwV4jJPSsd0gYNAvcSPik5J4hR70LKsoYkCpMxlWFTJ9bzkzl7Peeh/4ioAAl hyPg== X-Gm-Message-State: AC+VfDypEuzlohmhzQvdncdWFaMN3KqeBAMjZHJTS9aP6/ZHM1Ve//6D 1gLZ1gsxlwcqslFoyQRMgkq8j4YekueFatvr/93CYJTvRl59dGtxPxtvJ/lnkHlQgzUxQYqvDAB f18Ci+NxErZQQ9I4z+J0ukQn/UD9wf8tzn1Cu/BdfLdaNwUI+74ds2Lc/7buNS0Sh6e95AMjP90 U= X-Received: by 2002:adf:df0b:0:b0:306:8f5b:1b49 with SMTP id y11-20020adfdf0b000000b003068f5b1b49mr1422304wrl.47.1684345563253; Wed, 17 May 2023 10:46:03 -0700 (PDT) X-Google-Smtp-Source: ACHHUZ6lF0UgXidq71/eH8Qx9VS0MtYRlQxdm6VlDro4ydt2eoGgFo5fEacE9PwVsyHAh4ZGosCN1A== X-Received: by 2002:adf:df0b:0:b0:306:8f5b:1b49 with SMTP id y11-20020adfdf0b000000b003068f5b1b49mr1422282wrl.47.1684345562812; Wed, 17 May 2023 10:46:02 -0700 (PDT) Received: from [192.168.10.118] ([2001:b07:6468:f312:9af8:e5f5:7516:fa89]) by smtp.gmail.com with ESMTPSA id m9-20020a7bca49000000b003f3157988f8sm2902494wml.26.2023.05.17.10.46.01 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 17 May 2023 10:46:01 -0700 (PDT) From: Paolo Bonzini To: qemu-devel@nongnu.org Cc: John Snow Subject: [PULL 24/68] mkvenv: add --diagnose option to explain "ensure" failures Date: Wed, 17 May 2023 19:44:36 +0200 Message-Id: <20230517174520.887405-25-pbonzini@redhat.com> X-Mailer: git-send-email 2.40.1 In-Reply-To: <20230517174520.887405-1-pbonzini@redhat.com> References: <20230517174520.887405-1-pbonzini@redhat.com> MIME-Version: 1.0 Received-SPF: pass client-ip=170.10.133.124; envelope-from=pbonzini@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org From: John Snow This is a routine that is designed to print some usable info for human beings back out to the terminal if/when "mkvenv ensure" fails to locate or install a package during configure time, such as meson or sphinx. Since we are requiring that "meson" and "sphinx" are installed to the same Python environment as QEMU is configured to build with, this can produce some surprising failures when things are mismatched. This method is here to try and ease that sting by offering some actionable diagnosis. Signed-off-by: John Snow Message-Id: <20230511035435.734312-8-jsnow@redhat.com> Signed-off-by: Paolo Bonzini --- python/scripts/mkvenv.py | 170 ++++++++++++++++++++++++++++++++++++++- 1 file changed, 169 insertions(+), 1 deletion(-) diff --git a/python/scripts/mkvenv.py b/python/scripts/mkvenv.py index f6c4f37f84dd..2dbbc7020b48 100644 --- a/python/scripts/mkvenv.py +++ b/python/scripts/mkvenv.py @@ -51,6 +51,8 @@ import logging import os from pathlib import Path +import re +import shutil import site import subprocess import sys @@ -60,6 +62,7 @@ Any, Optional, Sequence, + Tuple, Union, ) import venv @@ -331,6 +334,128 @@ def _stringify(data: Union[str, bytes]) -> str: print(builder.get_value("env_exe")) +def pkgname_from_depspec(dep_spec: str) -> str: + """ + Parse package name out of a PEP-508 depspec. + + See https://peps.python.org/pep-0508/#names + """ + match = re.match( + r"^([A-Z0-9]([A-Z0-9._-]*[A-Z0-9])?)", dep_spec, re.IGNORECASE + ) + if not match: + raise ValueError( + f"dep_spec '{dep_spec}'" + " does not appear to contain a valid package name" + ) + return match.group(0) + + +def diagnose( + dep_spec: str, + online: bool, + wheels_dir: Optional[Union[str, Path]], + prog: Optional[str], +) -> Tuple[str, bool]: + """ + Offer a summary to the user as to why a package failed to be installed. + + :param dep_spec: The package we tried to ensure, e.g. 'meson>=0.61.5' + :param online: Did we allow PyPI access? + :param prog: + Optionally, a shell program name that can be used as a + bellwether to detect if this program is installed elsewhere on + the system. This is used to offer advice when a program is + detected for a different python version. + :param wheels_dir: + Optionally, a directory that was searched for vendored packages. + """ + # pylint: disable=too-many-branches + + # Some errors are not particularly serious + bad = False + + pkg_name = pkgname_from_depspec(dep_spec) + pkg_version = None + + has_importlib = False + try: + # Python 3.8+ stdlib + # pylint: disable=import-outside-toplevel + # pylint: disable=no-name-in-module + # pylint: disable=import-error + from importlib.metadata import ( # type: ignore + PackageNotFoundError, + version, + ) + + has_importlib = True + try: + pkg_version = version(pkg_name) + except PackageNotFoundError: + pass + except ModuleNotFoundError: + pass + + lines = [] + + if pkg_version: + lines.append( + f"Python package '{pkg_name}' version '{pkg_version}' was found," + " but isn't suitable." + ) + elif has_importlib: + lines.append( + f"Python package '{pkg_name}' was not found nor installed." + ) + else: + lines.append( + f"Python package '{pkg_name}' is either not found or" + " not a suitable version." + ) + + if wheels_dir: + lines.append( + "No suitable version found in, or failed to install from" + f" '{wheels_dir}'." + ) + bad = True + + if online: + lines.append("A suitable version could not be obtained from PyPI.") + bad = True + else: + lines.append( + "mkvenv was configured to operate offline and did not check PyPI." + ) + + if prog and not pkg_version: + which = shutil.which(prog) + if which: + if sys.base_prefix in site.PREFIXES: + pypath = Path(sys.executable).resolve() + lines.append( + f"'{prog}' was detected on your system at '{which}', " + f"but the Python package '{pkg_name}' was not found by " + f"this Python interpreter ('{pypath}'). " + f"Typically this means that '{prog}' has been installed " + "against a different Python interpreter on your system." + ) + else: + lines.append( + f"'{prog}' was detected on your system at '{which}', " + "but the build is using an isolated virtual environment." + ) + bad = True + + lines = [f" • {line}" for line in lines] + if bad: + lines.insert(0, f"Could not provide build dependency '{dep_spec}':") + else: + lines.insert(0, f"'{dep_spec}' not found:") + return os.linesep.join(lines), bad + + def pip_install( args: Sequence[str], online: bool = False, @@ -364,7 +489,7 @@ def pip_install( ) -def ensure( +def _do_ensure( dep_specs: Sequence[str], online: bool = False, wheels_dir: Optional[Union[str, Path]] = None, @@ -402,6 +527,39 @@ def ensure( pip_install(args=absent, online=online, wheels_dir=wheels_dir) +def ensure( + dep_specs: Sequence[str], + online: bool = False, + wheels_dir: Optional[Union[str, Path]] = None, + prog: Optional[str] = None, +) -> None: + """ + Use pip to ensure we have the package specified by @dep_specs. + + If the package is already installed, do nothing. If online and + wheels_dir are both provided, prefer packages found in wheels_dir + first before connecting to PyPI. + + :param dep_specs: + PEP 508 dependency specifications. e.g. ['meson>=0.61.5']. + :param online: If True, fall back to PyPI. + :param wheels_dir: If specified, search this path for packages. + :param prog: + If specified, use this program name for error diagnostics that will + be presented to the user. e.g., 'sphinx-build' can be used as a + bellwether for the presence of 'sphinx'. + """ + print(f"mkvenv: checking for {', '.join(dep_specs)}", file=sys.stderr) + try: + _do_ensure(dep_specs, online, wheels_dir) + except subprocess.CalledProcessError as exc: + # Well, that's not good. + msg, bad = diagnose(dep_specs[0], online, wheels_dir, prog) + if bad: + raise Ouch(msg) from exc + raise SystemExit(f"\n{msg}\n\n") from exc + + def _add_create_subcommand(subparsers: Any) -> None: subparser = subparsers.add_parser("create", help="create a venv") subparser.add_argument( @@ -427,6 +585,15 @@ def _add_ensure_subcommand(subparsers: Any) -> None: action="store", help="Path to vendored packages where we may install from.", ) + subparser.add_argument( + "--diagnose", + type=str, + action="store", + help=( + "Name of a shell utility to use for " + "diagnostics if this command fails." + ), + ) subparser.add_argument( "dep_specs", type=str, @@ -476,6 +643,7 @@ def main() -> int: dep_specs=args.dep_specs, online=args.online, wheels_dir=args.dir, + prog=args.diagnose, ) logger.debug("mkvenv.py %s: exiting", args.command) except Ouch as exc: From patchwork Wed May 17 17:44:37 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Paolo Bonzini X-Patchwork-Id: 1782854 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=nongnu.org (client-ip=209.51.188.17; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=) Authentication-Results: legolas.ozlabs.org; dkim=pass (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=aBk/qPg7; dkim-atps=neutral Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-ECDSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4QM1240vRmz20KF for ; Thu, 18 May 2023 03:54:04 +1000 (AEST) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1pzLGJ-00041f-4f; Wed, 17 May 2023 13:48:32 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pzLE3-0007Zy-HS for qemu-devel@nongnu.org; Wed, 17 May 2023 13:46:13 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.129.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pzLE1-0004GR-LX for qemu-devel@nongnu.org; Wed, 17 May 2023 13:46:11 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1684345568; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=/8sa5ey1iXBczVQw05fbQpg967ju52FxIa+kQUUZh3o=; b=aBk/qPg7JO/xo5CeaTG1NiTdGPwFYJDyJ3jJCz9LXg2y2C+lEFUKqX+YgmWlwbzq/fLumY 9hHdj7fQvSloEnabDH5VGc+Vu3JhUvAdqByX/I4gV1wbT4Oc8eQF1g5nEnr39rwJfBC73v WQz3acJjayxsCJxV5DycWNz+/OcPE78= Received: from mail-wm1-f69.google.com (mail-wm1-f69.google.com [209.85.128.69]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-619-olZbirinOU6JqH6WuqwEVA-1; Wed, 17 May 2023 13:46:07 -0400 X-MC-Unique: olZbirinOU6JqH6WuqwEVA-1 Received: by mail-wm1-f69.google.com with SMTP id 5b1f17b1804b1-3f4fb0193c6so4462735e9.1 for ; Wed, 17 May 2023 10:46:06 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1684345565; x=1686937565; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=/8sa5ey1iXBczVQw05fbQpg967ju52FxIa+kQUUZh3o=; b=J5Panmc6WgDlE5QdTZLZCUVt7PTiHQ2SU+Elm1yuhA+Vm397KdKoUB3mvrKKpieBEp HTfxt4DdwA97C58H+CQ3/3SV5ICVz0ElOPwxb/W618AKnsvwCxteos3nKHaJQ3twlOOQ IDq7mHE06X3QDdbyrSRn7n04ZmQcaxqSdVR50Dary6aN/4Z/qDbFuO++s1am27QMpjHf IcWK9pgNQqRuU/uuYXCrFJJ+tqqtvOjr4Yjb9eksKbtiCed/OQ7f+LMFfbpmYq5N0HZw JjdCKCm5UYkihr/8np5StHbEmV/Zpo1MCjrF+prX1/lFIyst69IbZS47iv//n6gb4hxm BY1w== X-Gm-Message-State: AC+VfDzSWQeD1UYl+guICNRfG+ZO0kcAgQ9OtgWPdqc+gAQefILLUnHJ 53OifrimdJWoRmi+ehXoNwQlww0TJxHJxCVz1EaApzcIosfDg7OLdU1kGaFHfDFILWi2dUddUni 6WJDXKLSqwvGRrCTFe708oocgkcFLTJ5fQWLyjqWGfVUqCnN59srRLl5mgQjs1elyVZpftl9aDP 0= X-Received: by 2002:a1c:cc16:0:b0:3f4:24c8:f7c6 with SMTP id h22-20020a1ccc16000000b003f424c8f7c6mr23619261wmb.16.1684345564784; Wed, 17 May 2023 10:46:04 -0700 (PDT) X-Google-Smtp-Source: ACHHUZ7OIEHteI3zSreRs4HyMRs08pQN8prg5X39/R3S+aZBS55uDoNpt4s2dUr+tH510AQgvyKTJg== X-Received: by 2002:a1c:cc16:0:b0:3f4:24c8:f7c6 with SMTP id h22-20020a1ccc16000000b003f424c8f7c6mr23619245wmb.16.1684345564462; Wed, 17 May 2023 10:46:04 -0700 (PDT) Received: from [192.168.10.118] ([2001:b07:6468:f312:9af8:e5f5:7516:fa89]) by smtp.gmail.com with ESMTPSA id c16-20020a7bc850000000b003f42d3111b8sm2895163wml.30.2023.05.17.10.46.03 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 17 May 2023 10:46:03 -0700 (PDT) From: Paolo Bonzini To: qemu-devel@nongnu.org Cc: John Snow Subject: [PULL 25/68] mkvenv: add console script entry point generation Date: Wed, 17 May 2023 19:44:37 +0200 Message-Id: <20230517174520.887405-26-pbonzini@redhat.com> X-Mailer: git-send-email 2.40.1 In-Reply-To: <20230517174520.887405-1-pbonzini@redhat.com> References: <20230517174520.887405-1-pbonzini@redhat.com> MIME-Version: 1.0 Received-SPF: pass client-ip=170.10.129.124; envelope-from=pbonzini@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org From: John Snow When creating a virtual environment that inherits system packages, script entry points (like "meson", "sphinx-build", etc) are not re-generated with the correct shebang. When you are *inside* of the venv, this is not a problem, but if you are *outside* of it, you will not have a script that engages the virtual environment appropriately. Add a mechanism that generates new entry points for pre-existing packages so that we can use these scripts to run "meson", "sphinx-build", "pip", unambiguously inside the venv. Signed-off-by: Paolo Bonzini Signed-off-by: John Snow Message-Id: <20230511035435.734312-9-jsnow@redhat.com> Signed-off-by: Paolo Bonzini --- python/scripts/mkvenv.py | 114 +++++++++++++++++++++++++++++++++++++++ python/setup.cfg | 3 ++ 2 files changed, 117 insertions(+) diff --git a/python/scripts/mkvenv.py b/python/scripts/mkvenv.py index 2dbbc7020b48..f17c3d3606ba 100644 --- a/python/scripts/mkvenv.py +++ b/python/scripts/mkvenv.py @@ -60,6 +60,7 @@ from types import SimpleNamespace from typing import ( Any, + Iterator, Optional, Sequence, Tuple, @@ -69,6 +70,7 @@ import warnings import distlib.database +import distlib.scripts import distlib.version @@ -334,6 +336,113 @@ def _stringify(data: Union[str, bytes]) -> str: print(builder.get_value("env_exe")) +def _gen_importlib(packages: Sequence[str]) -> Iterator[str]: + # pylint: disable=import-outside-toplevel + # pylint: disable=no-name-in-module + # pylint: disable=import-error + try: + # First preference: Python 3.8+ stdlib + from importlib.metadata import ( # type: ignore + PackageNotFoundError, + distribution, + ) + except ImportError as exc: + logger.debug("%s", str(exc)) + # Second preference: Commonly available PyPI backport + from importlib_metadata import ( # type: ignore + PackageNotFoundError, + distribution, + ) + + def _generator() -> Iterator[str]: + for package in packages: + try: + entry_points = distribution(package).entry_points + except PackageNotFoundError: + continue + + # The EntryPoints type is only available in 3.10+, + # treat this as a vanilla list and filter it ourselves. + entry_points = filter( + lambda ep: ep.group == "console_scripts", entry_points + ) + + for entry_point in entry_points: + yield f"{entry_point.name} = {entry_point.value}" + + return _generator() + + +def _gen_pkg_resources(packages: Sequence[str]) -> Iterator[str]: + # pylint: disable=import-outside-toplevel + # Bundled with setuptools; has a good chance of being available. + import pkg_resources + + def _generator() -> Iterator[str]: + for package in packages: + try: + eps = pkg_resources.get_entry_map(package, "console_scripts") + except pkg_resources.DistributionNotFound: + continue + + for entry_point in eps.values(): + yield str(entry_point) + + return _generator() + + +def generate_console_scripts( + packages: Sequence[str], + python_path: Optional[str] = None, + bin_path: Optional[str] = None, +) -> None: + """ + Generate script shims for console_script entry points in @packages. + """ + if python_path is None: + python_path = sys.executable + if bin_path is None: + bin_path = sysconfig.get_path("scripts") + assert bin_path is not None + + logger.debug( + "generate_console_scripts(packages=%s, python_path=%s, bin_path=%s)", + packages, + python_path, + bin_path, + ) + + if not packages: + return + + def _get_entry_points() -> Iterator[str]: + """Python 3.7 compatibility shim for iterating entry points.""" + # Python 3.8+, or Python 3.7 with importlib_metadata installed. + try: + return _gen_importlib(packages) + except ImportError as exc: + logger.debug("%s", str(exc)) + + # Python 3.7 with setuptools installed. + try: + return _gen_pkg_resources(packages) + except ImportError as exc: + logger.debug("%s", str(exc)) + raise Ouch( + "Neither importlib.metadata nor pkg_resources found, " + "can't generate console script shims.\n" + "Use Python 3.8+, or install importlib-metadata or setuptools." + ) from exc + + maker = distlib.scripts.ScriptMaker(None, bin_path) + maker.variants = {""} + maker.clobber = False + + for entry_point in _get_entry_points(): + for filename in maker.make(entry_point): + logger.debug("wrote console_script '%s'", filename) + + def pkgname_from_depspec(dep_spec: str) -> str: """ Parse package name out of a PEP-508 depspec. @@ -512,6 +621,7 @@ def _do_ensure( ) dist_path = distlib.database.DistributionPath(include_egg=True) absent = [] + present = [] for spec in dep_specs: matcher = distlib.version.LegacyMatcher(spec) dist = dist_path.get_distribution(matcher.name) @@ -519,6 +629,10 @@ def _do_ensure( absent.append(spec) else: logger.info("found %s", dist) + present.append(matcher.name) + + if present: + generate_console_scripts(present) if absent: # Some packages are missing or aren't a suitable version, diff --git a/python/setup.cfg b/python/setup.cfg index d680374b2950..826a2771ba5d 100644 --- a/python/setup.cfg +++ b/python/setup.cfg @@ -119,6 +119,9 @@ ignore_missing_imports = True [mypy-distlib.database] ignore_missing_imports = True +[mypy-distlib.scripts] +ignore_missing_imports = True + [mypy-distlib.version] ignore_missing_imports = True From patchwork Wed May 17 17:44:38 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Paolo Bonzini X-Patchwork-Id: 1782843 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=nongnu.org (client-ip=209.51.188.17; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=) Authentication-Results: legolas.ozlabs.org; dkim=pass (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=Vm9vkZpu; dkim-atps=neutral Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-ECDSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4QM0ws6JfPz20dq for ; Thu, 18 May 2023 03:49:33 +1000 (AEST) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1pzLET-0008JR-Bw; Wed, 17 May 2023 13:46:40 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pzLE4-0007ay-DK for qemu-devel@nongnu.org; Wed, 17 May 2023 13:46:13 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pzLE2-0004Gc-Nq for qemu-devel@nongnu.org; Wed, 17 May 2023 13:46:12 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1684345569; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=WaYDq55qknnU8QMRXQL79aAwtuV8ftUSiPH6BPevf0s=; b=Vm9vkZpuuJF7WYKYvr6wyyZH06UAuxdoKF5qqzTcIG9X04IydA9afNgBucJ6+1SrhC4r0w uGODZEIq7I6gaSFvSpQHUHIftcTYKgS4O7a+D4y8nLpTEH2g3I3URkeCbsivjJvESwcVQu 5mxYDMxBdewAoAeIutiUFO9Sy52KfXs= Received: from mail-wm1-f72.google.com (mail-wm1-f72.google.com [209.85.128.72]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-10-NGa0J2yCP8ioLEW79B3ahw-1; Wed, 17 May 2023 13:46:08 -0400 X-MC-Unique: NGa0J2yCP8ioLEW79B3ahw-1 Received: by mail-wm1-f72.google.com with SMTP id 5b1f17b1804b1-3f433a2308bso48200955e9.0 for ; Wed, 17 May 2023 10:46:07 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1684345566; x=1686937566; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=WaYDq55qknnU8QMRXQL79aAwtuV8ftUSiPH6BPevf0s=; b=IgxjKoQfctJ7uexwfkocoVMn5MmHQ7gofXPLwJP++K5UWFT05Bj5F6ivm/8Vvt16xq QPmvKU2aKTMpXK87XXyw7eama3L8wso+RZ1LRxdvLve4sOApJk7IHDYIQLoBBthmxnnY CU2GH/GdGOACKUqz6pwS3Th/FkJcVTQQevMtDra/M2pOpbu1cq6ZcxhtbeuFBCRbmAap 0vgwd9fgGgkBz2qCFVm5cki37bMROLYm2TeBek0oubFQHnZtpTsDnJ/u0FfsNnA11SGI JRY6Np7A585KRSlVRf45bcp8N8ynHfqSyj6S7+2znMbAf6pRkciAHHv9bb9UGkBdwn5q WYAw== X-Gm-Message-State: AC+VfDwloKVt89UdYRG4It8m3q4J2UNADe2xzm/iJKHGS2nvHsB63jYL F45MSXZd6R7KzxCyg4EE8w9HAPkwN//+654jp54q0F67E3vNqgqnVRZZJxTwIpuFl9Jx3qA+N+C /dKxVZ42gXxjXZL+G+EKHK9QJNIfwx2y93BDjcqkzXaGHEoobcngZk3v5D3HenM08fjdS15LvDq Q= X-Received: by 2002:a5d:5390:0:b0:309:43fe:1b9d with SMTP id d16-20020a5d5390000000b0030943fe1b9dmr1214808wrv.29.1684345566556; Wed, 17 May 2023 10:46:06 -0700 (PDT) X-Google-Smtp-Source: ACHHUZ53fv4UeWQam2w4euBseJK2xSLdTTcdMOZvg6DMmaF2K286raysJbe0OZrCAYWk+4QvKQ6rsg== X-Received: by 2002:a5d:5390:0:b0:309:43fe:1b9d with SMTP id d16-20020a5d5390000000b0030943fe1b9dmr1214798wrv.29.1684345566255; Wed, 17 May 2023 10:46:06 -0700 (PDT) Received: from [192.168.10.118] ([2001:b07:6468:f312:9af8:e5f5:7516:fa89]) by smtp.gmail.com with ESMTPSA id m39-20020a05600c3b2700b003f07ef4e3e0sm5341583wms.0.2023.05.17.10.46.04 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 17 May 2023 10:46:05 -0700 (PDT) From: Paolo Bonzini To: qemu-devel@nongnu.org Cc: John Snow Subject: [PULL 26/68] mkvenv: use pip's vendored distlib as a fallback Date: Wed, 17 May 2023 19:44:38 +0200 Message-Id: <20230517174520.887405-27-pbonzini@redhat.com> X-Mailer: git-send-email 2.40.1 In-Reply-To: <20230517174520.887405-1-pbonzini@redhat.com> References: <20230517174520.887405-1-pbonzini@redhat.com> MIME-Version: 1.0 Received-SPF: pass client-ip=170.10.133.124; envelope-from=pbonzini@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org From: John Snow distlib is usually not installed on Linux distribution, but it is vendored into pip. Because the virtual environment has pip via ensurepip, we can piggy-back on pip's vendored version. This could break if they move our cheese in the future, but the fix would be simply to require distlib. If it is debundled, as it is on msys, it is simply available directly. Signed-off-by: John Snow [Move to toplevel. - Paolo] Signed-off-by: Paolo Bonzini --- python/scripts/mkvenv.py | 25 ++++++++++++++++++++++--- python/setup.cfg | 18 ++++++++++++++++++ 2 files changed, 40 insertions(+), 3 deletions(-) diff --git a/python/scripts/mkvenv.py b/python/scripts/mkvenv.py index f17c3d3606ba..fb91f922d2b9 100644 --- a/python/scripts/mkvenv.py +++ b/python/scripts/mkvenv.py @@ -69,10 +69,25 @@ import venv import warnings -import distlib.database -import distlib.scripts -import distlib.version +# Try to load distlib, with a fallback to pip's vendored version. +# HAVE_DISTLIB is checked below, just-in-time, so that mkvenv does not fail +# outside the venv or before a potential call to ensurepip in checkpip(). +HAVE_DISTLIB = True +try: + import distlib.database + import distlib.scripts + import distlib.version +except ImportError: + try: + # Reach into pip's cookie jar. pylint and flake8 don't understand + # that these imports will be used via distlib.xxx. + from pip._vendor import distlib + import pip._vendor.distlib.database # noqa, pylint: disable=unused-import + import pip._vendor.distlib.scripts # noqa, pylint: disable=unused-import + import pip._vendor.distlib.version # noqa, pylint: disable=unused-import + except ImportError: + HAVE_DISTLIB = False # Do not add any mandatory dependencies from outside the stdlib: # This script *must* be usable standalone! @@ -664,6 +679,10 @@ def ensure( bellwether for the presence of 'sphinx'. """ print(f"mkvenv: checking for {', '.join(dep_specs)}", file=sys.stderr) + + if not HAVE_DISTLIB: + raise Ouch("a usable distlib could not be found, please install it") + try: _do_ensure(dep_specs, online, wheels_dir) except subprocess.CalledProcessError as exc: diff --git a/python/setup.cfg b/python/setup.cfg index 826a2771ba5d..fc3fae5b1076 100644 --- a/python/setup.cfg +++ b/python/setup.cfg @@ -125,6 +125,24 @@ ignore_missing_imports = True [mypy-distlib.version] ignore_missing_imports = True +[mypy-pip] +ignore_missing_imports = True + +[mypy-pip._vendor] +ignore_missing_imports = True + +[mypy-pip._vendor.distlib] +ignore_missing_imports = True + +[mypy-pip._vendor.distlib.database] +ignore_missing_imports = True + +[mypy-pip._vendor.distlib.scripts] +ignore_missing_imports = True + +[mypy-pip._vendor.distlib.version] +ignore_missing_imports = True + [pylint.messages control] # Disable the message, report, category or checker with the given id(s). You # can either give multiple identifiers separated by comma (,) or put this From patchwork Wed May 17 17:44:39 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Paolo Bonzini X-Patchwork-Id: 1782863 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=nongnu.org (client-ip=209.51.188.17; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=) Authentication-Results: legolas.ozlabs.org; dkim=pass (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=KQmw0EFf; dkim-atps=neutral Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-ECDSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4QM14b60K2z20dX for ; Thu, 18 May 2023 03:56:15 +1000 (AEST) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1pzLED-0008F5-9Y; Wed, 17 May 2023 13:46:21 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pzLE6-0007eE-CM for qemu-devel@nongnu.org; Wed, 17 May 2023 13:46:15 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pzLE4-0004H6-Fq for qemu-devel@nongnu.org; Wed, 17 May 2023 13:46:14 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1684345572; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=3IN3vpyWO0eLzaYR0wBWQOI2Q29IOE8Vh96ivvWrrr8=; b=KQmw0EFf9WIz4rM//fzRG9u9ZUBHJX2H2jqCMCT4YPDiQg2g5yq2bNK4uRK9btVHAsdmds bGJMr/yhulO9u4IBOsUZOJJUqqAdydhlVMB7ld6/1i8+E60jdYmDaknw0Tinfaxf/G4fD2 AnZ5j/WuY5uniduFtnl5KA1jKgL3Mhs= Received: from mail-wm1-f72.google.com (mail-wm1-f72.google.com [209.85.128.72]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-142-cApT83a9PKW4nRKENi5Nsw-1; Wed, 17 May 2023 13:46:10 -0400 X-MC-Unique: cApT83a9PKW4nRKENi5Nsw-1 Received: by mail-wm1-f72.google.com with SMTP id 5b1f17b1804b1-3f41a04a297so4361895e9.3 for ; Wed, 17 May 2023 10:46:09 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1684345568; x=1686937568; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=3IN3vpyWO0eLzaYR0wBWQOI2Q29IOE8Vh96ivvWrrr8=; b=EEAAbjYQsFd3O87d7SYYtVPLNCuT+zjFdRUmaD8e2IIm7ryCjDXsh0WXkjGJLbRJLp w8tkr2DEbRu3bbFqlscgGaEYT2Qm07dJ0qa1Tw7F2HAD4CI3XOpz5NCR9mlISmkV577p OqePGU9F5yMYxzhgZV63TKcGSbbTGVxTl8X4kj0qkOKxUXEFhgSkxa9QdHm2aEETB3Id CyqqJfw/Gk1RwY8tSft+xWRDrvsTmM8GZysrlCRCUTL6ESiLjRlKfeAvHPEPBD2SHWJS 5SvdXBG5RJTvSYFtxU1uA5bWmqSUyc3ewu/iI6MvyxQrSgV3egoLcQqNkyygTdK8r2f8 LhEg== X-Gm-Message-State: AC+VfDzBUFqgTwjPX5HQobDT8sybD0ODXbpHjzihX79Wja0sf693P2Gk zDjVydlLAaGKsRW7KQ2sjIs0qo9Ko5MBs51iFKjOptTIyAcs7eyFimn+poR4YkawrR2VcBgRYzb wNpIdgDRqrTcYjxeuOl4Nl2gyiN4eZNSupxxKiPYbGWSFzEt9UW3OjkNQCkLPt3XUG1nJOEuA6J M= X-Received: by 2002:a05:600c:2059:b0:3f4:2492:a91f with SMTP id p25-20020a05600c205900b003f42492a91fmr25003479wmg.27.1684345568420; Wed, 17 May 2023 10:46:08 -0700 (PDT) X-Google-Smtp-Source: ACHHUZ5pTxnHgPnjekIOSuQ4zSf8iwj79v10T4rk46Sp7LaTnWVpRIYaxVwFR2wMfF14X5Z0TY5o2Q== X-Received: by 2002:a05:600c:2059:b0:3f4:2492:a91f with SMTP id p25-20020a05600c205900b003f42492a91fmr25003471wmg.27.1684345568117; Wed, 17 May 2023 10:46:08 -0700 (PDT) Received: from [192.168.10.118] ([2001:b07:6468:f312:9af8:e5f5:7516:fa89]) by smtp.gmail.com with ESMTPSA id o4-20020a5d4a84000000b002c54c9bd71fsm3390013wrq.93.2023.05.17.10.46.06 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 17 May 2023 10:46:07 -0700 (PDT) From: Paolo Bonzini To: qemu-devel@nongnu.org Cc: John Snow Subject: [PULL 27/68] mkvenv: avoid ensurepip if pip is installed Date: Wed, 17 May 2023 19:44:39 +0200 Message-Id: <20230517174520.887405-28-pbonzini@redhat.com> X-Mailer: git-send-email 2.40.1 In-Reply-To: <20230517174520.887405-1-pbonzini@redhat.com> References: <20230517174520.887405-1-pbonzini@redhat.com> MIME-Version: 1.0 Received-SPF: pass client-ip=170.10.133.124; envelope-from=pbonzini@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org From: John Snow Signed-off-by: Paolo Bonzini --- python/scripts/mkvenv.py | 67 ++++++++++++++++++++++++++++++++++++++-- 1 file changed, 64 insertions(+), 3 deletions(-) diff --git a/python/scripts/mkvenv.py b/python/scripts/mkvenv.py index fb91f922d2b9..a9c566029e6f 100644 --- a/python/scripts/mkvenv.py +++ b/python/scripts/mkvenv.py @@ -11,6 +11,8 @@ Commands: command Description create create a venv + post_init + post-venv initialization ensure Ensure that the specified package is installed. -------------------------------------------------- @@ -25,6 +27,13 @@ -------------------------------------------------- +usage: mkvenv post_init [-h] + +options: + -h, --help show this help message and exit + +-------------------------------------------------- + usage: mkvenv ensure [-h] [--online] [--dir DIR] dep_spec... positional arguments: @@ -111,7 +120,9 @@ class QemuEnvBuilder(venv.EnvBuilder): The primary difference is that it emulates a "nested" virtual environment when invoked from inside of an existing virtual - environment by including packages from the parent. + environment by including packages from the parent. Also, + "ensurepip" is replaced if possible with just recreating pip's + console_scripts inside the virtual environment. Parameters for base class init: - system_site_packages: bool = False @@ -138,8 +149,19 @@ def __init__(self, *args: Any, **kwargs: Any) -> None: # The venv we are currently in, also does so. kwargs["system_site_packages"] = sys.base_prefix in site.PREFIXES - if kwargs.get("with_pip", False): - check_ensurepip() + # ensurepip is slow: venv creation can be very fast for cases where + # we allow the use of system_site_packages. Therefore, ensurepip is + # replaced with our own script generation once the virtual environment + # is setup. + self.want_pip = kwargs.get("with_pip", False) + if self.want_pip: + if ( + kwargs.get("system_site_packages", False) + and not need_ensurepip() + ): + kwargs["with_pip"] = False + else: + check_ensurepip() super().__init__(*args, **kwargs) @@ -211,6 +233,14 @@ def post_post_setup(self, context: SimpleNamespace) -> None: with open(pth_file, "w", encoding="UTF-8") as file: file.write(parent_libpath + os.linesep) + if self.want_pip: + args = [ + context.env_exe, + __file__, + "post_init", + ] + subprocess.run(args, check=True) + def get_value(self, field: str) -> str: """ Get a string value from the context namespace after a call to build. @@ -223,6 +253,19 @@ def get_value(self, field: str) -> str: return ret +def need_ensurepip() -> bool: + """ + Tests for the presence of setuptools and pip. + + :return: `True` if we do not detect both packages. + """ + # Don't try to actually import them, it's fraught with danger: + # https://github.com/pypa/setuptools/issues/2993 + if find_spec("setuptools") and find_spec("pip"): + return False + return True + + def check_ensurepip() -> None: """ Check that we have ensurepip. @@ -693,6 +736,17 @@ def ensure( raise SystemExit(f"\n{msg}\n\n") from exc +def post_venv_setup() -> None: + """ + This is intended to be run *inside the venv* after it is created. + """ + logger.debug("post_venv_setup()") + # Generate a 'pip' script so the venv is usable in a normal + # way from the CLI. This only happens when we inherited pip from a + # parent/system-site and haven't run ensurepip in some way. + generate_console_scripts(["pip"]) + + def _add_create_subcommand(subparsers: Any) -> None: subparser = subparsers.add_parser("create", help="create a venv") subparser.add_argument( @@ -703,6 +757,10 @@ def _add_create_subcommand(subparsers: Any) -> None: ) +def _add_post_init_subcommand(subparsers: Any) -> None: + subparsers.add_parser("post_init", help="post-venv initialization") + + def _add_ensure_subcommand(subparsers: Any) -> None: subparser = subparsers.add_parser( "ensure", help="Ensure that the specified package is installed." @@ -761,6 +819,7 @@ def main() -> int: ) _add_create_subcommand(subparsers) + _add_post_init_subcommand(subparsers) _add_ensure_subcommand(subparsers) args = parser.parse_args() @@ -771,6 +830,8 @@ def main() -> int: system_site_packages=True, clear=True, ) + if args.command == "post_init": + post_venv_setup() if args.command == "ensure": ensure( dep_specs=args.dep_specs, From patchwork Wed May 17 17:44:40 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Paolo Bonzini X-Patchwork-Id: 1782833 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=nongnu.org (client-ip=209.51.188.17; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=) Authentication-Results: legolas.ozlabs.org; dkim=pass (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=LPuOuvAW; dkim-atps=neutral Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-ECDSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4QM0tJ21lBz20dq for ; Thu, 18 May 2023 03:47:20 +1000 (AEST) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1pzLEB-00081J-5M; Wed, 17 May 2023 13:46:19 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pzLE7-0007hL-NO for qemu-devel@nongnu.org; Wed, 17 May 2023 13:46:16 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pzLE5-0004HP-V8 for qemu-devel@nongnu.org; Wed, 17 May 2023 13:46:15 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1684345573; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=tlEc08hsHfooO7rQikJ5l6wNa9dPNHYDs4dAyWzQnhQ=; b=LPuOuvAW7kXitgpLkQ7kin+dTCx0a2W13HxoJszDbtLEotNo1ahD9Huvzb8sVdU5a2nVe+ AimXIjDMklReGqsVNY9qNiYs54WnSZqqPKDk6W7H8ep8JcLpXeAX9J51tpoR6+fY3WIqZT /w8yAtFa9mRKBGisNi63stIJmJEMT8c= Received: from mail-wm1-f71.google.com (mail-wm1-f71.google.com [209.85.128.71]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-480-zZOHHL5lM86OqkTXTXsohw-1; Wed, 17 May 2023 13:46:12 -0400 X-MC-Unique: zZOHHL5lM86OqkTXTXsohw-1 Received: by mail-wm1-f71.google.com with SMTP id 5b1f17b1804b1-3f42867b47dso6595495e9.2 for ; Wed, 17 May 2023 10:46:11 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1684345570; x=1686937570; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=tlEc08hsHfooO7rQikJ5l6wNa9dPNHYDs4dAyWzQnhQ=; b=FeP6VVvcYEvYf37OR5PP75U/srzZx9JANU0RyNNwvmB5hIf1CDvasNVR0f+ikP8x1y QVPMLtGhoBUWsBNIpchgD5vhbOe4D0/Bymw6fHDgJFasGmMmOUn12lrApwMIvTBZK6UL kX8DtEDrXCfCwsJcWW4Kqowj0MnQusDfPhjvi4A8Ewq+SnXl6qvxmshPYpwHwtNbqFPP +AqIZbko1A9HOlfOHMrwq3pOf+m5DUMbsiFbf6vBp4dCiyKCYZOG2uu2Zk0gH3sappj6 QQxm5Uv25cs5O+je+q3wTunYJUdzEZDI8cKKSs5U3LNdGBthvVIg8FJyetnPn5NYWC5x JYvg== X-Gm-Message-State: AC+VfDy5Fb/4aM0GYl1JCRnR5DCZAXdW1HPfFqHTtXqojiOLCXYj3o7s L8T5tihSwy6LdtF73GozO0Qje9WUXutfLGns7F9XHpomBcV5BPgn4Pdi0EPxy8oXMInOztAfukD RxpSj9kGr4/Ns7T/7rRymKpeHVhrcZV70N6ilNFNyFriFo5+TPNJvBnC20phq/S0pJaKdB9Sh7H U= X-Received: by 2002:a7b:c5d6:0:b0:3f1:969f:c9d0 with SMTP id n22-20020a7bc5d6000000b003f1969fc9d0mr30995808wmk.4.1684345570281; Wed, 17 May 2023 10:46:10 -0700 (PDT) X-Google-Smtp-Source: ACHHUZ7C9McPTBZwF21I//SvuoO91O5rjbWK50HlGFvBj2d7RK4TrP4siB1PcrSlQVwNkKyl1Z+rbA== X-Received: by 2002:a7b:c5d6:0:b0:3f1:969f:c9d0 with SMTP id n22-20020a7bc5d6000000b003f1969fc9d0mr30995785wmk.4.1684345569717; Wed, 17 May 2023 10:46:09 -0700 (PDT) Received: from [192.168.10.118] ([2001:b07:6468:f312:9af8:e5f5:7516:fa89]) by smtp.gmail.com with ESMTPSA id z4-20020a05600c220400b003f4069417absm2849765wml.24.2023.05.17.10.46.08 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 17 May 2023 10:46:09 -0700 (PDT) From: Paolo Bonzini To: qemu-devel@nongnu.org Cc: John Snow Subject: [PULL 28/68] mkvenv: work around broken pip installations on Debian 10 Date: Wed, 17 May 2023 19:44:40 +0200 Message-Id: <20230517174520.887405-29-pbonzini@redhat.com> X-Mailer: git-send-email 2.40.1 In-Reply-To: <20230517174520.887405-1-pbonzini@redhat.com> References: <20230517174520.887405-1-pbonzini@redhat.com> MIME-Version: 1.0 Received-SPF: pass client-ip=170.10.133.124; envelope-from=pbonzini@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org From: John Snow This is a workaround intended for Debian 10, where the debian-patched pip does not function correctly if accessed from within a virtual environment. We don't support Debian 10 as a build platform any longer, though we do still utilize it for our build-tricore-softmmu CI test. It's also possible that this bug might appear on other derivative platforms and this workaround may prove useful. Signed-off-by: Paolo Bonzini Signed-off-by: John Snow Message-Id: <20230511035435.734312-11-jsnow@redhat.com> Signed-off-by: Paolo Bonzini --- python/scripts/mkvenv.py | 72 +++++++++++++++++++++++++++++++--------- 1 file changed, 56 insertions(+), 16 deletions(-) diff --git a/python/scripts/mkvenv.py b/python/scripts/mkvenv.py index a9c566029e6f..558619910efd 100644 --- a/python/scripts/mkvenv.py +++ b/python/scripts/mkvenv.py @@ -161,7 +161,7 @@ def __init__(self, *args: Any, **kwargs: Any) -> None: ): kwargs["with_pip"] = False else: - check_ensurepip() + check_ensurepip(suggest_remedy=True) super().__init__(*args, **kwargs) @@ -266,7 +266,7 @@ def need_ensurepip() -> bool: return True -def check_ensurepip() -> None: +def check_ensurepip(prefix: str = "", suggest_remedy: bool = False) -> None: """ Check that we have ensurepip. @@ -277,12 +277,15 @@ def check_ensurepip() -> None: "Python's ensurepip module is not found.\n" "It's normally part of the Python standard library, " "maybe your distribution packages it separately?\n" - "Either install ensurepip, or alleviate the need for it in the " - "first place by installing pip and setuptools for " - f"'{sys.executable}'.\n" - "(Hint: Debian puts ensurepip in its python3-venv package.)" + "(Debian puts ensurepip in its python3-venv package.)\n" ) - raise Ouch(msg) + if suggest_remedy: + msg += ( + "Either install ensurepip, or alleviate the need for it in the" + " first place by installing pip and setuptools for " + f"'{sys.executable}'.\n" + ) + raise Ouch(prefix + msg) # ensurepip uses pyexpat, which can also go missing on us: if not find_spec("pyexpat"): @@ -290,12 +293,15 @@ def check_ensurepip() -> None: "Python's pyexpat module is not found.\n" "It's normally part of the Python standard library, " "maybe your distribution packages it separately?\n" - "Either install pyexpat, or alleviate the need for it in the " - "first place by installing pip and setuptools for " - f"'{sys.executable}'.\n\n" - "(Hint: NetBSD's pkgsrc debundles this to e.g. 'py310-expat'.)" + "(NetBSD's pkgsrc debundles this to e.g. 'py310-expat'.)\n" ) - raise Ouch(msg) + if suggest_remedy: + msg += ( + "Either install pyexpat, or alleviate the need for it in the " + "first place by installing pip and setuptools for " + f"'{sys.executable}'.\n" + ) + raise Ouch(prefix + msg) def make_venv( # pylint: disable=too-many-arguments @@ -501,6 +507,38 @@ def _get_entry_points() -> Iterator[str]: logger.debug("wrote console_script '%s'", filename) +def checkpip() -> bool: + """ + Debian10 has a pip that's broken when used inside of a virtual environment. + + We try to detect and correct that case here. + """ + try: + # pylint: disable=import-outside-toplevel,unused-import,import-error + # pylint: disable=redefined-outer-name + import pip._internal # type: ignore # noqa: F401 + + logger.debug("pip appears to be working correctly.") + return False + except ModuleNotFoundError as exc: + if exc.name == "pip._internal": + # Uh, fair enough. They did say "internal". + # Let's just assume it's fine. + return False + logger.warning("pip appears to be malfunctioning: %s", str(exc)) + + check_ensurepip("pip appears to be non-functional, and ") + + logger.debug("Attempting to repair pip ...") + subprocess.run( + (sys.executable, "-m", "ensurepip"), + stdout=subprocess.DEVNULL, + check=True, + ) + logger.debug("Pip is now (hopefully) repaired!") + return True + + def pkgname_from_depspec(dep_spec: str) -> str: """ Parse package name out of a PEP-508 depspec. @@ -741,10 +779,12 @@ def post_venv_setup() -> None: This is intended to be run *inside the venv* after it is created. """ logger.debug("post_venv_setup()") - # Generate a 'pip' script so the venv is usable in a normal - # way from the CLI. This only happens when we inherited pip from a - # parent/system-site and haven't run ensurepip in some way. - generate_console_scripts(["pip"]) + # Test for a broken pip (Debian 10 or derivative?) and fix it if needed + if not checkpip(): + # Finally, generate a 'pip' script so the venv is usable in a normal + # way from the CLI. This only happens when we inherited pip from a + # parent/system-site and haven't run ensurepip in some way. + generate_console_scripts(["pip"]) def _add_create_subcommand(subparsers: Any) -> None: From patchwork Wed May 17 17:44:41 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Paolo Bonzini X-Patchwork-Id: 1782876 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=nongnu.org (client-ip=209.51.188.17; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=) Authentication-Results: legolas.ozlabs.org; dkim=pass (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=KraDh1cf; dkim-atps=neutral Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-ECDSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4QM18l2bwPz20dX for ; Thu, 18 May 2023 03:59:51 +1000 (AEST) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1pzLEb-0000lW-4u; Wed, 17 May 2023 13:46:45 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pzLE9-0007mn-0X for qemu-devel@nongnu.org; Wed, 17 May 2023 13:46:17 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pzLE7-0004Hg-5d for qemu-devel@nongnu.org; Wed, 17 May 2023 13:46:16 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1684345574; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=gagIyj+ohvBxTGckPPYIFm5V3ZHBPriTT/5noq/lqAw=; b=KraDh1cfOLqjGrScnzHeW5+Lw+S5uSVJIZon9iETiXw/1cB/OL7DDP4OiBsMc5pzg+Tqia pEqVexXV1rvcZpckSzl0Re3hzEgf00b0btNDClKuqVs1P85DslPg2fyZCYKxE9llRoBM5S VhhfWhD4EJNX70SxgCO4iiyEfJqymTY= Received: from mail-wm1-f69.google.com (mail-wm1-f69.google.com [209.85.128.69]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-248-GCwVbUYuPCKPRHZnCKxdrg-1; Wed, 17 May 2023 13:46:13 -0400 X-MC-Unique: GCwVbUYuPCKPRHZnCKxdrg-1 Received: by mail-wm1-f69.google.com with SMTP id 5b1f17b1804b1-3f4245ffb03so6963025e9.3 for ; Wed, 17 May 2023 10:46:13 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1684345572; x=1686937572; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=gagIyj+ohvBxTGckPPYIFm5V3ZHBPriTT/5noq/lqAw=; b=ZO5E1KiXGT8qtN0Kof2Lag72LABHz6dzCiz90iuH02bOnqVeRrmVIqbleGGUQe7p/z FoAsyL57tHL2bFr0t+0nJCTNyYFIv3u+b7j55inBIv4rZp/aQrVOQm9AOpgQsXYoGtqZ D3C8pHKLwqYO68JsQY2HndRJvp9cl7LwzdmWxrjQAeZSXW2hgbejv3DxyyObXSBMc4yz aySn4hgNs7B5AngvCGDsxf6jYPvWmWs5o2mN5U7hzvuhBYQ6TIbgbf4Xxni1OrBtKNsl Vh7KwYNbUW8rrKEoxx2BUxqvggplABu5YP0RBESSlu7Pp4dBSlmP1knqAMUZUuE+mqXI teiA== X-Gm-Message-State: AC+VfDwuFKR6izaaBaWY16Yr1UC9xZ2dbCKW8fMghPktnG3Ap68E5F9b CtpHIl8kmSvsnyxux89BMInF6YuGidl/KQ+OWohNx30mr2eWNSboUM4imv7s9k3DA2UI+sJPw5k 3c5+oF81Dm75wtVPo37PXDmhZYY0JqoBrr+Xemjc/LeCxiQpMrbaE4st2cNV4Th9QC6HLnVq2Qc c= X-Received: by 2002:a05:600c:b4d:b0:3f4:253f:c3d9 with SMTP id k13-20020a05600c0b4d00b003f4253fc3d9mr22313250wmr.25.1684345571780; Wed, 17 May 2023 10:46:11 -0700 (PDT) X-Google-Smtp-Source: ACHHUZ4Ud8wd/NgkrqMF7GGJXFch6Ul9+xLqTnzu9pAxGOtlLs3G3hx3QOt1D7yY/jXZkQgqxDUQuQ== X-Received: by 2002:a05:600c:b4d:b0:3f4:253f:c3d9 with SMTP id k13-20020a05600c0b4d00b003f4253fc3d9mr22313238wmr.25.1684345571459; Wed, 17 May 2023 10:46:11 -0700 (PDT) Received: from [192.168.10.118] ([2001:b07:6468:f312:9af8:e5f5:7516:fa89]) by smtp.gmail.com with ESMTPSA id z26-20020a7bc7da000000b003f50d6ee334sm2832355wmk.47.2023.05.17.10.46.10 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 17 May 2023 10:46:10 -0700 (PDT) From: Paolo Bonzini To: qemu-devel@nongnu.org Cc: John Snow , =?utf-8?q?Daniel_P_=2E_Berrang=C3=A9?= Subject: [PULL 29/68] tests/docker: add python3-venv dependency Date: Wed, 17 May 2023 19:44:41 +0200 Message-Id: <20230517174520.887405-30-pbonzini@redhat.com> X-Mailer: git-send-email 2.40.1 In-Reply-To: <20230517174520.887405-1-pbonzini@redhat.com> References: <20230517174520.887405-1-pbonzini@redhat.com> MIME-Version: 1.0 Received-SPF: pass client-ip=170.10.133.124; envelope-from=pbonzini@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org From: John Snow Several debian-based tests need the python3-venv dependency as a consequence of Debian debundling the "ensurepip" module normally included with Python. As mkvenv.py stands as of this commit, Debian requires EITHER: (A) setuptools and pip, or (B) ensurepip mkvenv is a few seconds faster if you have setuptools and pip, so developers should prefer the first requirement. For the purposes of CI, the time-save is a wash; it's only a matter of who is responsible for installing pip and when; the timing is about the same. Arbitrarily, I chose adding ensurepip to the test configuration because it is normally part of the Python stdlib, and always having it allows us a more consistent cross-platform environment. Signed-off-by: John Snow Reviewed-by: Daniel P. BerrangĂ© Message-Id: <20230511035435.734312-12-jsnow@redhat.com> Signed-off-by: Paolo Bonzini --- tests/docker/dockerfiles/debian-all-test-cross.docker | 3 ++- tests/docker/dockerfiles/debian-hexagon-cross.docker | 3 ++- tests/docker/dockerfiles/debian-riscv64-cross.docker | 3 ++- tests/docker/dockerfiles/debian-tricore-cross.docker | 3 ++- 4 files changed, 8 insertions(+), 4 deletions(-) diff --git a/tests/docker/dockerfiles/debian-all-test-cross.docker b/tests/docker/dockerfiles/debian-all-test-cross.docker index 981e9bdc7b0e..f9f401544a05 100644 --- a/tests/docker/dockerfiles/debian-all-test-cross.docker +++ b/tests/docker/dockerfiles/debian-all-test-cross.docker @@ -57,7 +57,8 @@ RUN DEBIAN_FRONTEND=noninteractive eatmydata \ gcc-sh4-linux-gnu \ libc6-dev-sh4-cross \ gcc-sparc64-linux-gnu \ - libc6-dev-sparc64-cross + libc6-dev-sparc64-cross \ + python3-venv ENV QEMU_CONFIGURE_OPTS --disable-system --disable-docs --disable-tools ENV DEF_TARGET_LIST aarch64-linux-user,alpha-linux-user,arm-linux-user,hppa-linux-user,i386-linux-user,m68k-linux-user,mips-linux-user,mips64-linux-user,mips64el-linux-user,mipsel-linux-user,ppc-linux-user,ppc64-linux-user,ppc64le-linux-user,riscv64-linux-user,s390x-linux-user,sh4-linux-user,sparc64-linux-user diff --git a/tests/docker/dockerfiles/debian-hexagon-cross.docker b/tests/docker/dockerfiles/debian-hexagon-cross.docker index b99d99f94365..c2cfb6a5d0fa 100644 --- a/tests/docker/dockerfiles/debian-hexagon-cross.docker +++ b/tests/docker/dockerfiles/debian-hexagon-cross.docker @@ -20,7 +20,8 @@ RUN apt-get update && \ bison \ flex \ git \ - ninja-build && \ + ninja-build \ + python3-venv && \ # Install QEMU build deps for use in CI DEBIAN_FRONTEND=noninteractive eatmydata \ apt build-dep -yy --arch-only qemu diff --git a/tests/docker/dockerfiles/debian-riscv64-cross.docker b/tests/docker/dockerfiles/debian-riscv64-cross.docker index 803afb957345..081404e01410 100644 --- a/tests/docker/dockerfiles/debian-riscv64-cross.docker +++ b/tests/docker/dockerfiles/debian-riscv64-cross.docker @@ -28,7 +28,8 @@ RUN DEBIAN_FRONTEND=noninteractive eatmydata apt install -yy \ libglib2.0-dev \ ninja-build \ pkg-config \ - python3 + python3 \ + python3-venv # Add ports and riscv64 architecture RUN echo "deb http://ftp.ports.debian.org/debian-ports/ sid main" >> /etc/apt/sources.list diff --git a/tests/docker/dockerfiles/debian-tricore-cross.docker b/tests/docker/dockerfiles/debian-tricore-cross.docker index cfd2faf9a803..269bfa8d4230 100644 --- a/tests/docker/dockerfiles/debian-tricore-cross.docker +++ b/tests/docker/dockerfiles/debian-tricore-cross.docker @@ -33,7 +33,8 @@ RUN apt update && \ pkgconf \ python3-pip \ python3-setuptools \ - python3-wheel + python3-wheel \ + python3-venv RUN curl -#SL https://github.com/bkoppelmann/package_940/releases/download/tricore-toolchain-9.40/tricore-toolchain-9.4.0.tar.gz \ | tar -xzC /usr/local/ From patchwork Wed May 17 17:44:42 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Paolo Bonzini X-Patchwork-Id: 1782832 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=nongnu.org (client-ip=209.51.188.17; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=) Authentication-Results: legolas.ozlabs.org; dkim=pass (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=ScyNVaqO; dkim-atps=neutral Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-ECDSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4QM0tH6RqMz20dn for ; Thu, 18 May 2023 03:47:19 +1000 (AEST) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1pzLEc-0000tr-BC; Wed, 17 May 2023 13:46:46 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pzLEA-0007xS-Ob for qemu-devel@nongnu.org; Wed, 17 May 2023 13:46:18 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.129.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pzLE8-0004Hu-K8 for qemu-devel@nongnu.org; Wed, 17 May 2023 13:46:17 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1684345576; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=GKBaLoWGdZ0RkyXEDY1BFQu3+SJiQML2IvZoS6F7Z0o=; b=ScyNVaqOpF7LcJpg/K5WblayG3hFqIZdYvdFc7LSaB4o5GxfXGfubW8mOIB78vH66lr1ED g8idDUNVDsJvQIA1JXZhpsZmOnN4/tOsJBIS2msivgsNnAPu1kUcR1dutd4KgvALlDHRM6 P2fAkczAyfxfK48OOuCRw2l0tjd2z9U= Received: from mail-wm1-f71.google.com (mail-wm1-f71.google.com [209.85.128.71]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-459-7e4fBP5oPsGReDbtn9-HFQ-1; Wed, 17 May 2023 13:46:14 -0400 X-MC-Unique: 7e4fBP5oPsGReDbtn9-HFQ-1 Received: by mail-wm1-f71.google.com with SMTP id 5b1f17b1804b1-3f426d4944fso4322815e9.1 for ; Wed, 17 May 2023 10:46:14 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1684345573; x=1686937573; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=GKBaLoWGdZ0RkyXEDY1BFQu3+SJiQML2IvZoS6F7Z0o=; b=JzjIkbmgTZEW3DGYDuRBkU1qprDYgIJF1FEGGwp1GTMhOBZowoyqI71gfA1jeeZ2zo XAp9stCYnzjAQ7A+VvbfR2FDc9E3waOmAPEHk+JxWZzURNbUPNkppTVMaqbF7TIzLrWf /OlCc8trGp/BZDl4WNb8J24t5iV3owuLFsRSpajlg9eqQWN/SOdLQBSmNxVlF6yVjf1L OsnNviZU5zUBW6XX299B35MNj2K1NPkhYgbGniWGM1rjPkpbsJUqP238QCQfLqll7VTi y/AxerJX89/6NLFxhs73vBQaWf+8atj5M7J9us56EqIs5g7f8GRZ0GGAz5xWKDlweNN0 wk5w== X-Gm-Message-State: AC+VfDyHRZl6MISrRXbrL1keKV5IKHYv+KwW49j32HFMpx5Vj5Mkf1uR OK/+hRxvlCergkwol1IWRXa2qgIS5RIq3A/iD6yAzDpC11RvuFoQCp+/RwKmWwF5c8sTO6R0k9g VKBnV3eST9pKYr1/D2Cjr3uTddq/uH0Blm87kyZKLyQW9m62XEo1PDGHceNOKjCfqcSp/BUqNBB 4= X-Received: by 2002:a05:600c:2313:b0:3f4:2266:8cf0 with SMTP id 19-20020a05600c231300b003f422668cf0mr23818606wmo.10.1684345573375; Wed, 17 May 2023 10:46:13 -0700 (PDT) X-Google-Smtp-Source: ACHHUZ46qK7StXan5/bvyLrSVW/8o6eQMBdepyJm9zeLXtBxUk6shOoqkazbc9JWuCSoUd4G1miQlA== X-Received: by 2002:a05:600c:2313:b0:3f4:2266:8cf0 with SMTP id 19-20020a05600c231300b003f422668cf0mr23818595wmo.10.1684345573109; Wed, 17 May 2023 10:46:13 -0700 (PDT) Received: from [192.168.10.118] ([2001:b07:6468:f312:9af8:e5f5:7516:fa89]) by smtp.gmail.com with ESMTPSA id h2-20020a1ccc02000000b003f421979398sm2854775wmb.26.2023.05.17.10.46.11 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 17 May 2023 10:46:12 -0700 (PDT) From: Paolo Bonzini To: qemu-devel@nongnu.org Cc: John Snow , =?utf-8?q?Daniel_P_=2E_Berrang=C3=A9?= Subject: [PULL 30/68] tests/vm: Configure netbsd to use Python 3.10 Date: Wed, 17 May 2023 19:44:42 +0200 Message-Id: <20230517174520.887405-31-pbonzini@redhat.com> X-Mailer: git-send-email 2.40.1 In-Reply-To: <20230517174520.887405-1-pbonzini@redhat.com> References: <20230517174520.887405-1-pbonzini@redhat.com> MIME-Version: 1.0 Received-SPF: pass client-ip=170.10.129.124; envelope-from=pbonzini@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org From: John Snow NetBSD removes some packages from the Python stdlib, but only re-packages them for Python 3.10. Switch to using Python 3.10. Signed-off-by: John Snow Reviewed-by: Daniel P. BerrangĂ© Message-Id: <20230511035435.734312-13-jsnow@redhat.com> Signed-off-by: Paolo Bonzini --- tests/vm/netbsd | 1 + 1 file changed, 1 insertion(+) diff --git a/tests/vm/netbsd b/tests/vm/netbsd index 0b9536ca17bf..13eae109c042 100755 --- a/tests/vm/netbsd +++ b/tests/vm/netbsd @@ -30,6 +30,7 @@ class NetBSDVM(basevm.BaseVM): "git-base", "pkgconf", "xz", + "python310", "ninja-build", # gnu tools From patchwork Wed May 17 17:44:43 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Paolo Bonzini X-Patchwork-Id: 1782849 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=nongnu.org (client-ip=209.51.188.17; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=) Authentication-Results: legolas.ozlabs.org; dkim=pass (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=V1FHgvdt; dkim-atps=neutral Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-ECDSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4QM0zL59l7z20KF for ; Thu, 18 May 2023 03:51:42 +1000 (AEST) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1pzLGW-00043V-4w; Wed, 17 May 2023 13:48:45 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pzLEW-00009Z-Vj for qemu-devel@nongnu.org; Wed, 17 May 2023 13:46:41 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.129.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pzLER-0004Lk-1Q for qemu-devel@nongnu.org; Wed, 17 May 2023 13:46:40 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1684345594; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=kYwBoQfoPq8gzyxMAjFuE9dhgscDIFH9kaTNoAGSNNY=; b=V1FHgvdtU/ngN64inz5RE6YXcACvro7pnC9NHWExS5uSEzxjM49v3w+QmBGTbD75AMnVzb ulIBXTSvab62yKsYwCy3U8a5LdoUHnUj78augoyU6dY9/hL25ww7JK6EFVSblUYvz0FUMi m4HPjkvFDycHYDi7klnyEnKRYuHlM+g= Received: from mail-wr1-f72.google.com (mail-wr1-f72.google.com [209.85.221.72]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-39-72vcZyboO_6QyNr0qwUkng-1; Wed, 17 May 2023 13:46:20 -0400 X-MC-Unique: 72vcZyboO_6QyNr0qwUkng-1 Received: by mail-wr1-f72.google.com with SMTP id ffacd0b85a97d-30629b36d9bso471568f8f.0 for ; Wed, 17 May 2023 10:46:16 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1684345575; x=1686937575; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=kYwBoQfoPq8gzyxMAjFuE9dhgscDIFH9kaTNoAGSNNY=; b=IgtyPw3OYJDRjkz8gp7tjfjSo9JK1PWxZf2CyJhZ60PAsDMqTux/qAgsgkYAFkeiyX rxxG+ynEcKcSY/Yf7JTvUGTPJ13CYQdS6h+mXRNG7Cg8JwPx8sGASaPQP2JwnlePCZnl 6IyuCoC7N60vHUrjbu1hdn+VNNUSRhnbu+aLduJA3O3/SfRxZxk7/cFptZnptkAR+VKC e2+RZ2w6qyPF62MA0Li6EFHehWpbD0YsGZor2Y0eBrpUjyloX5CK2dLFb31IM7PMyIzb z+o/rzMQKzTE+bkifgTEJx1ORnU6pcR765xUvrcCG/ipTweleHwzaAHgZNXQTYwfdZ2x 45KQ== X-Gm-Message-State: AC+VfDyfiThSw+NBDXySDzNhTKKqAADAlqURrEGowgdvwzKOvWxgOOY5 GKO2PXXRi6G6hmnIjY26TY3mjLTv8gqqNum6iAK7IpOf/kSJxxXm7Iep0uMME189sCpLW7Esn6A IWJvHR9kusqY2TQAaRq3m88zPEYQWU7g6nrRmb1emc5+TxqwaIthtSN5PosPQZMmbyPcYGBg20B Y= X-Received: by 2002:a5d:5409:0:b0:309:3b8d:16a8 with SMTP id g9-20020a5d5409000000b003093b8d16a8mr1106139wrv.50.1684345574869; Wed, 17 May 2023 10:46:14 -0700 (PDT) X-Google-Smtp-Source: ACHHUZ43oBVghnPH+heaC9c21i6x4e9GytlK3+gmTauN/1EMHrl/hADM3ALyJSmJVsk5wOnsZE93ew== X-Received: by 2002:a5d:5409:0:b0:309:3b8d:16a8 with SMTP id g9-20020a5d5409000000b003093b8d16a8mr1106130wrv.50.1684345574553; Wed, 17 May 2023 10:46:14 -0700 (PDT) Received: from [192.168.10.118] ([2001:b07:6468:f312:9af8:e5f5:7516:fa89]) by smtp.gmail.com with ESMTPSA id h14-20020adffd4e000000b00304aba2cfcbsm3478111wrs.7.2023.05.17.10.46.13 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 17 May 2023 10:46:14 -0700 (PDT) From: Paolo Bonzini To: qemu-devel@nongnu.org Cc: John Snow , =?utf-8?q?Daniel_P_=2E_Berrang=C3=A9?= Subject: [PULL 31/68] tests/vm: add py310-expat to NetBSD Date: Wed, 17 May 2023 19:44:43 +0200 Message-Id: <20230517174520.887405-32-pbonzini@redhat.com> X-Mailer: git-send-email 2.40.1 In-Reply-To: <20230517174520.887405-1-pbonzini@redhat.com> References: <20230517174520.887405-1-pbonzini@redhat.com> MIME-Version: 1.0 Received-SPF: pass client-ip=170.10.129.124; envelope-from=pbonzini@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org From: John Snow NetBSD cannot successfully run "ensurepip" without access to the pyexpat module, which NetBSD debundles. Like the Debian patch, it would be strictly faster long term to install pip/setuptools, and I recommend developers at their workstations take that approach instead. For the purposes of a throwaway VM, there's not really a speed difference for who is responsible for installing pip; us (needs py310-pip) or Python (needs py310-expat). Signed-off-by: John Snow Reviewed-by: Daniel P. BerrangĂ© Message-Id: <20230511035435.734312-14-jsnow@redhat.com> Signed-off-by: Paolo Bonzini --- tests/vm/netbsd | 1 + 1 file changed, 1 insertion(+) diff --git a/tests/vm/netbsd b/tests/vm/netbsd index 13eae109c042..c7e3f1e7357f 100755 --- a/tests/vm/netbsd +++ b/tests/vm/netbsd @@ -31,6 +31,7 @@ class NetBSDVM(basevm.BaseVM): "pkgconf", "xz", "python310", + "py310-expat", "ninja-build", # gnu tools From patchwork Wed May 17 17:44:44 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Paolo Bonzini X-Patchwork-Id: 1782840 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=nongnu.org (client-ip=209.51.188.17; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=) Authentication-Results: legolas.ozlabs.org; dkim=pass (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=SCnMHYZ2; dkim-atps=neutral Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-ECDSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4QM0wq3Rvsz20dq for ; Thu, 18 May 2023 03:49:31 +1000 (AEST) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1pzLEc-0000xs-Q0; Wed, 17 May 2023 13:46:49 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pzLED-0008I1-Js for qemu-devel@nongnu.org; Wed, 17 May 2023 13:46:21 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pzLEC-0004IV-3K for qemu-devel@nongnu.org; Wed, 17 May 2023 13:46:21 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1684345579; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=s4rCHC/nblK5C3SukFSnNAPlnH8JVV43s/yQSX1o5NM=; b=SCnMHYZ2FGsB+WPXstluTFnrfUtPnryUbrcr+sZ4bL0jml0HuBHFqd5cJO+1Vz8S6t0SiT yB4NozxQ9nEWPvcFWVyBxI/XstHi8oYoxC5H/PJ/ExMiEygHZc43wOA0OKyo8B9rkO8mxn zOQez+3Mrh5TZ2OhUUk/Gg3DID3vJic= Received: from mail-wm1-f72.google.com (mail-wm1-f72.google.com [209.85.128.72]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-544-H7TLKsZpMamCXy7VPPTcrw-1; Wed, 17 May 2023 13:46:18 -0400 X-MC-Unique: H7TLKsZpMamCXy7VPPTcrw-1 Received: by mail-wm1-f72.google.com with SMTP id 5b1f17b1804b1-3f42867b47dso6596095e9.2 for ; Wed, 17 May 2023 10:46:17 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1684345576; x=1686937576; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=s4rCHC/nblK5C3SukFSnNAPlnH8JVV43s/yQSX1o5NM=; b=VrIHn6KtmwvDG1wQRhNRTPcTpmqi5AbpaxcFRTmGLEIZawNbYiGhiCGSgC2ADdD77C GnhCTBW9wCu3Bs5QSwlX0dSTC4R/fjPMkSOuWSVBlAvp0HkN2AzjGT+QERw/oex4f+hd qXuZrm4Fc6TGsXxCKRqUKZJwek20VArW6acpOU31XhoFCXcvaYKT5gLEgs/AfsFlFzoH h2e50JhnlWpnNuSg28kH0H/nBf5rIGVTj9cXKVCaC3jE7+d7KNNT3UQvqh9p4CcYWztt Ch5hQLt55FR8VNeu857Me70C0m50/Df0z/rIBf3Z4xsAAj3JIzAUucuc3z5XrvyZCv83 9KWQ== X-Gm-Message-State: AC+VfDxlbHqaPm4/y+XZ1X5KLgj7jZE/woDg2IUWoUzhA7BqdhQfCKVf jL/OeeKmr4IFbL2mK1QTE8qevFzybILVhKVjHOBbPheQW1s3CpEubhuf3U8hMr49XH4QUUUxrKk F8FBDtDq5gu3vVMvmS0gJuNxlsC/HYI07HMYEDkVs2MLkCPwCmidYiEaEo0uL48ZdQJznLV1Cpo E= X-Received: by 2002:a05:600c:20f:b0:3f4:2267:10c0 with SMTP id 15-20020a05600c020f00b003f4226710c0mr25905713wmi.28.1684345576508; Wed, 17 May 2023 10:46:16 -0700 (PDT) X-Google-Smtp-Source: ACHHUZ6ADDSZo0VOxhXhWuPDe5aNHzYXmi3IOHrHVyT2XwOTfB5JrZuwHYUmGuBRjxHdwzONUjrVSg== X-Received: by 2002:a05:600c:20f:b0:3f4:2267:10c0 with SMTP id 15-20020a05600c020f00b003f4226710c0mr25905694wmi.28.1684345576079; Wed, 17 May 2023 10:46:16 -0700 (PDT) Received: from [192.168.10.118] ([2001:b07:6468:f312:9af8:e5f5:7516:fa89]) by smtp.gmail.com with ESMTPSA id g6-20020a7bc4c6000000b003f195d2f1a9sm2902425wmk.15.2023.05.17.10.46.15 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 17 May 2023 10:46:15 -0700 (PDT) From: Paolo Bonzini To: qemu-devel@nongnu.org Cc: John Snow Subject: [PULL 32/68] python: add vendor.py utility Date: Wed, 17 May 2023 19:44:44 +0200 Message-Id: <20230517174520.887405-33-pbonzini@redhat.com> X-Mailer: git-send-email 2.40.1 In-Reply-To: <20230517174520.887405-1-pbonzini@redhat.com> References: <20230517174520.887405-1-pbonzini@redhat.com> MIME-Version: 1.0 Received-SPF: pass client-ip=170.10.133.124; envelope-from=pbonzini@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org From: John Snow This is a teeny-tiny script that just downloads any packages we want to vendor from PyPI and stores them in qemu.git/python/wheels/. If I'm hit by a meteor, it'll be easy to replicate what I have done in order to udpate the vendored source. We don't really care which python runs it; it exists as a meta-utility with no external dependencies and we won't package or install it. It will be monitored by the linters/type checkers, though; so it's guaranteed safe on python 3.6+. Signed-off-by: John Snow Message-Id: <20230511035435.734312-15-jsnow@redhat.com> Signed-off-by: Paolo Bonzini --- python/scripts/vendor.py | 74 ++++++++++++++++++++++++++++++++++++++++ 1 file changed, 74 insertions(+) create mode 100755 python/scripts/vendor.py diff --git a/python/scripts/vendor.py b/python/scripts/vendor.py new file mode 100755 index 000000000000..23708430ea68 --- /dev/null +++ b/python/scripts/vendor.py @@ -0,0 +1,74 @@ +#!/usr/bin/env python3 +""" +vendor - QEMU python vendoring utility + +usage: vendor [-h] + +QEMU python vendoring utility + +options: + -h, --help show this help message and exit +""" + +# Copyright (C) 2023 Red Hat, Inc. +# +# Authors: +# John Snow +# +# This work is licensed under the terms of the GNU GPL, version 2 or +# later. See the COPYING file in the top-level directory. + +import argparse +import os +from pathlib import Path +import subprocess +import sys +import tempfile + + +def main() -> int: + """Run the vendoring utility. See module-level docstring.""" + loud = False + if os.environ.get("DEBUG") or os.environ.get("V"): + loud = True + + # No options or anything for now, but I guess + # you'll figure that out when you run --help. + parser = argparse.ArgumentParser( + prog="vendor", + description="QEMU python vendoring utility", + ) + parser.parse_args() + + packages = { + "meson==0.61.5": + "58c2ddb5f885da0e929f15d89f38d8a7d97f981f56815bcba008414f8511f59a", + } + + vendor_dir = Path(__file__, "..", "..", "wheels").resolve() + + with tempfile.NamedTemporaryFile(mode="w", encoding="utf-8") as file: + for dep_spec, checksum in packages.items(): + file.write(f"{dep_spec} --hash=sha256:{checksum}") + file.flush() + + cli_args = [ + "pip", + "download", + "--dest", + str(vendor_dir), + "--require-hashes", + "-r", + file.name, + ] + if loud: + cli_args.append("-v") + + print(" ".join(cli_args)) + subprocess.run(cli_args, check=True) + + return 0 + + +if __name__ == "__main__": + sys.exit(main()) From patchwork Wed May 17 17:44:45 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Paolo Bonzini X-Patchwork-Id: 1782882 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=nongnu.org (client-ip=209.51.188.17; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=) Authentication-Results: legolas.ozlabs.org; dkim=pass (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=Iw945ZwT; dkim-atps=neutral Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-ECDSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4QM1Dh0wdlz20dn for ; Thu, 18 May 2023 04:03:16 +1000 (AEST) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1pzLEa-0000i6-U0; Wed, 17 May 2023 13:46:44 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pzLEF-0008Jg-3a for qemu-devel@nongnu.org; Wed, 17 May 2023 13:46:33 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pzLED-0004J1-Jt for qemu-devel@nongnu.org; Wed, 17 May 2023 13:46:22 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1684345581; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=ktxuVoeCftsXoioRTwebbxI5P4dwKOFzWaEcjYs2LFM=; b=Iw945ZwTVbX1QaA1eU3RFGu+8lF4Byr3sUdh8G49aby9nVHHbJ/SYSPBk2o9AEVoQ3MPrH 2y05rTMNfFY7tNTFuu2G2bvCpk2ZyCe4BRyiwm1r5ziqkS3g1y43W8fV9Olhne0BLxHt1A OOAyyj8ILOE9pcmmntOzfDULQgCnCVc= Received: from mail-wm1-f72.google.com (mail-wm1-f72.google.com [209.85.128.72]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-556-b3jMMQWaM82NvAA1_Kwhjg-1; Wed, 17 May 2023 13:46:19 -0400 X-MC-Unique: b3jMMQWaM82NvAA1_Kwhjg-1 Received: by mail-wm1-f72.google.com with SMTP id 5b1f17b1804b1-3f42b226871so4969275e9.0 for ; Wed, 17 May 2023 10:46:19 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1684345578; x=1686937578; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=ktxuVoeCftsXoioRTwebbxI5P4dwKOFzWaEcjYs2LFM=; b=PyNAjJC4Qq9k0aKoWJZh1IDc/uiPfLO2DxAICadKtTQyXD9nv6MixEc0n/f4YjvTkg xLiw4KAEg3EMqblGZnDmFHYK72Uf1Dzm0IW+4nu2iuXQPXjoMic3Cn881FR2nzzYG/hX AOXXAEq6VIfGqlPl9XzwMrgkaVbP1NsCHlh0iK8wH74JJ4GsZsfmcpYX3pn0MM7IRl5i lPCozBFifgtEwqHUo2uob8JwYqT6f1DnWaqbbQ1aXwZ+CiObKlBARrF8McaGMs94kBi/ QU0j/Kr5zhtPALuIjhHbj3JgnBK4Z7OAewN9jkrzQMm6BDZ6hO4CY/WMQ1asBbD7nZLZ 4A3Q== X-Gm-Message-State: AC+VfDwMYA4nM69/hNtInownGwNpIczvl+QMlyhqCNdZch+Zb3NKXyKH iFYCl7Og8kl1898vGO4o9s2b/mZLTBGahxVdpJR7wdaVVpglDGfLlGVhRDqWtUP2Hc+U21f8HBm mCPPCrdDCN7C2LH2pNG5jo2T1bm5lBr0jaVPmRV9SGolVklFwDNuLiwACTy0SIQAFAEb43JDYjl 8= X-Received: by 2002:a05:600c:3787:b0:3f4:2bcd:5a6e with SMTP id o7-20020a05600c378700b003f42bcd5a6emr20206702wmr.30.1684345578028; Wed, 17 May 2023 10:46:18 -0700 (PDT) X-Google-Smtp-Source: ACHHUZ4gFcnkxAvYj5CRIMh2ADqAV05VSrdfz4/VN5fkmL1lCv+BXdYe7MqA8hT1vZFl0IhmBokfFw== X-Received: by 2002:a05:600c:3787:b0:3f4:2bcd:5a6e with SMTP id o7-20020a05600c378700b003f42bcd5a6emr20206688wmr.30.1684345577704; Wed, 17 May 2023 10:46:17 -0700 (PDT) Received: from [192.168.10.118] ([2001:b07:6468:f312:9af8:e5f5:7516:fa89]) by smtp.gmail.com with ESMTPSA id z2-20020a05600c220200b003f42cc7aac4sm2885044wml.37.2023.05.17.10.46.16 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 17 May 2023 10:46:17 -0700 (PDT) From: Paolo Bonzini To: qemu-devel@nongnu.org Cc: John Snow Subject: [PULL 33/68] configure: create a python venv unconditionally Date: Wed, 17 May 2023 19:44:45 +0200 Message-Id: <20230517174520.887405-34-pbonzini@redhat.com> X-Mailer: git-send-email 2.40.1 In-Reply-To: <20230517174520.887405-1-pbonzini@redhat.com> References: <20230517174520.887405-1-pbonzini@redhat.com> MIME-Version: 1.0 Received-SPF: pass client-ip=170.10.133.124; envelope-from=pbonzini@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org From: John Snow This patch changes the configure script so that it always creates and uses a python virtual environment unconditionally. Meson bootstrapping is temporarily altered to force the use of meson from git or vendored source (as packaged in our source tarballs). A subsequent commit restores the use of distribution-vendored Meson. Signed-off-by: John Snow Message-Id: <20230511035435.734312-16-jsnow@redhat.com> Signed-off-by: Paolo Bonzini --- configure | 34 +++++++++++++++++++++++++++++----- 1 file changed, 29 insertions(+), 5 deletions(-) diff --git a/configure b/configure index 243e2e0a0dbc..1d7db92ee3f9 100755 --- a/configure +++ b/configure @@ -625,7 +625,6 @@ check_py_version() { python= first_python= if test -z "${PYTHON}"; then - explicit_python=no # A bare 'python' is traditionally python 2.x, but some distros # have it as python 3.x, so check in both places. for binary in python3 python python3.11 python3.10 python3.9 python3.8 python3.7 python3.6; do @@ -644,7 +643,6 @@ else # Same as above, but only check the environment variable. has "${PYTHON}" || error_exit "The PYTHON environment variable does not point to an executable" python=$(command -v "$PYTHON") - explicit_python=yes if check_py_version "$python"; then # This one is good. first_python= @@ -729,7 +727,7 @@ for opt do ;; --install=*) ;; - --python=*) python="$optarg" ; explicit_python=yes + --python=*) python="$optarg" ;; --skip-meson) skip_meson=yes ;; @@ -1090,8 +1088,34 @@ if ! check_py_version "$python"; then "Use --python=/path/to/python to specify a supported Python." fi -# Resolve PATH + suppress writing compiled files -python="$(command -v "$python") -B" +# Resolve PATH +python="$(command -v "$python")" +explicit_python=yes + +# Create a Python virtual environment using our configured python. +# The stdout of this script will be the location of a symlink that +# points to the configured Python. +# Entry point scripts for pip, meson, and sphinx are generated if those +# packages are present. + +# Defaults assumed for now: +# - venv is cleared if it exists already; +# - venv is allowed to use system packages; +# - all setup is performed **offline**; +# - No packages are installed by default; +# - pip is not installed into the venv when possible, +# but ensurepip is called as a fallback when necessary. + +echo "python determined to be '$python'" +echo "python version: $($python --version)" + +python="$($python -B "${source_path}/python/scripts/mkvenv.py" create pyvenv)" +if test "$?" -ne 0 ; then + error_exit "python venv creation failed" +fi + +# Suppress writing compiled files +python="$python -B" has_meson() { local python_dir=$(dirname "$python") From patchwork Wed May 17 17:44:47 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Paolo Bonzini X-Patchwork-Id: 1782873 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=nongnu.org (client-ip=209.51.188.17; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=) Authentication-Results: legolas.ozlabs.org; dkim=pass (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=LDuZgMqk; dkim-atps=neutral Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-ECDSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4QM17w1Q8tz20dX for ; Thu, 18 May 2023 03:59:08 +1000 (AEST) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1pzLF4-0002IQ-GE; Wed, 17 May 2023 13:47:14 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pzLEJ-0008Tk-EQ for qemu-devel@nongnu.org; Wed, 17 May 2023 13:46:35 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.129.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pzLEH-0004K0-Eo for qemu-devel@nongnu.org; Wed, 17 May 2023 13:46:27 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1684345584; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=Mbxl8QbqXsMqvYzVdZ6VdOCeDAaqOga28d8VP6vA3vI=; b=LDuZgMqkdp+BthHl7uQgZbp98tyl7MjjGXcGlTVoRdeawiUzj9CE9537e6YmnZ4F+8EGJd V5Schirl1+6rxu2vl6M7taMaSwYAbTrDf3GC7F3a2iUwDj8gfdnVGuKg/IGtulodIzO2yq /IZgctwpX0ekymaUFzeBVodHT3TDrTM= Received: from mail-wr1-f69.google.com (mail-wr1-f69.google.com [209.85.221.69]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-97-nx8c8YbZNVukxsbDteRv-Q-1; Wed, 17 May 2023 13:46:22 -0400 X-MC-Unique: nx8c8YbZNVukxsbDteRv-Q-1 Received: by mail-wr1-f69.google.com with SMTP id ffacd0b85a97d-3064d0b726fso442253f8f.0 for ; Wed, 17 May 2023 10:46:22 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1684345581; x=1686937581; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=Mbxl8QbqXsMqvYzVdZ6VdOCeDAaqOga28d8VP6vA3vI=; b=fGkVW4akK85X1se6IrSMSUyBC9aO8JyXiDY3QIbKyBdnNtBVABG0ECF4XBf66Kck2o a5/YiIVcmjGA/2Za8Xdsv6Q+bsvmJEqpHi+o5gHPS6vgmAeH+xW674N8n9HJ5sgTihKb Y7UN4m/fSVA9xgvpi+C6/Sm94hPs57wkrFVdH1WQus3mSLSFF1qNrpvssNdDJBruu/lm msfAFGizftLOgMQ1C33XGfonWIl9GAoQIToakr1NZehxskr5NGUe//bnCNun8PNqqN11 /pVfx5VKtQK3V0zbfORftAnRYth2vmvXcYco9Qv06mvaUWHoJn/65BLtolN2xtOx94d1 T90g== X-Gm-Message-State: AC+VfDwZ/v5pVoTL0Zao6bzdRnTd7wLS0KrEM1pqoOib1oimSFJsbv/k EUeAhjsajYmlGss0kMD+y6GhWiYqMHErnwa9kz13zH/tHDg5MLLT+bmIRxr3GbSz0MUELjwHwP4 ngUmPa0c3QRwaPzJgVBAGFQIYJf5pB6HWp3gK+OEJkupSD1ID+YZf5/as6HmgA0/2DUQdaIaH7F A= X-Received: by 2002:adf:f452:0:b0:2f4:4e1a:bea2 with SMTP id f18-20020adff452000000b002f44e1abea2mr1090839wrp.59.1684345581274; Wed, 17 May 2023 10:46:21 -0700 (PDT) X-Google-Smtp-Source: ACHHUZ6RA3EwZS9m8x0fOwAQBRZwGImnDMW1TeFi8nnQN9FLRzEEb7a4pELeY7u/IQxf9V0voIvQHg== X-Received: by 2002:adf:f452:0:b0:2f4:4e1a:bea2 with SMTP id f18-20020adff452000000b002f44e1abea2mr1090825wrp.59.1684345580867; Wed, 17 May 2023 10:46:20 -0700 (PDT) Received: from [192.168.10.118] ([2001:b07:6468:f312:9af8:e5f5:7516:fa89]) by smtp.gmail.com with ESMTPSA id n14-20020adff08e000000b002fed865c55esm3343919wro.56.2023.05.17.10.46.19 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 17 May 2023 10:46:20 -0700 (PDT) From: Paolo Bonzini To: qemu-devel@nongnu.org Cc: John Snow Subject: [PULL 35/68] configure: use 'mkvenv ensure meson' to bootstrap meson Date: Wed, 17 May 2023 19:44:47 +0200 Message-Id: <20230517174520.887405-36-pbonzini@redhat.com> X-Mailer: git-send-email 2.40.1 In-Reply-To: <20230517174520.887405-1-pbonzini@redhat.com> References: <20230517174520.887405-1-pbonzini@redhat.com> MIME-Version: 1.0 Received-SPF: pass client-ip=170.10.129.124; envelope-from=pbonzini@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org From: John Snow This commit changes how we detect and install meson. It notably removes '--meson='. Currently, configure creates a lightweight Python virtual environment unconditionally using the user's configured $python that inherits system packages. Temporarily, we forced the use of meson source present via git submodule or in the release tarball. With this patch, we restore the ability to use a system-provided meson: If Meson is installed in the build venv and meets our minimum version requirements, we will use that Meson. This includes a system provided meson, which would be visible via system-site packages inside the venv. In the event that Meson is installed but *not for the chosen Python interpreter*, not found, or of insufficient version, we will attempt to install Meson from vendored source into the newly created Python virtual environment. This vendored installation replaces both the git submodule and tarball source mechanisms for sourcing meson. As a result of this patch, the Python interpreter we use for both our own build scripts *and* Meson extensions are always known to be the exact same Python. As a further benefit, there will also be a symlink available in the build directory that points to the correct, configured python and can be used by e.g. manual tests to invoke the correct, configured Python unambiguously. Signed-off-by: John Snow Message-Id: <20230511035435.734312-18-jsnow@redhat.com> Signed-off-by: Paolo Bonzini --- .gitlab-ci.d/buildtest-template.yml | 4 +- configure | 65 +++++------------------------ 2 files changed, 13 insertions(+), 56 deletions(-) diff --git a/.gitlab-ci.d/buildtest-template.yml b/.gitlab-ci.d/buildtest-template.yml index a6cfe9be970f..7edb50b76035 100644 --- a/.gitlab-ci.d/buildtest-template.yml +++ b/.gitlab-ci.d/buildtest-template.yml @@ -12,12 +12,12 @@ - mkdir build - cd build - ../configure --enable-werror --disable-docs --enable-fdt=system - ${LD_JOBS:+--meson=git} ${TARGETS:+--target-list="$TARGETS"} + ${TARGETS:+--target-list="$TARGETS"} $CONFIGURE_ARGS || { cat config.log meson-logs/meson-log.txt && exit 1; } - if test -n "$LD_JOBS"; then - ../meson/meson.py configure . -Dbackend_max_links="$LD_JOBS" ; + pyvenv/bin/meson configure . -Dbackend_max_links="$LD_JOBS" ; fi || exit 1; - make -j"$JOBS" - if test -n "$MAKE_CHECK_ARGS"; diff --git a/configure b/configure index 1d7db92ee3f9..4519c4d412f8 100755 --- a/configure +++ b/configure @@ -289,7 +289,6 @@ bsd_user="" pie="" coroutine="" plugins="$default_feature" -meson="" ninja="" bindir="bin" skip_meson=no @@ -731,8 +730,6 @@ for opt do ;; --skip-meson) skip_meson=yes ;; - --meson=*) meson="$optarg" - ;; --ninja=*) ninja="$optarg" ;; --smbd=*) smbd="$optarg" @@ -1017,7 +1014,6 @@ Advanced options (experts only): --cross-prefix-ARCH=PREFIX cross compiler prefix when building ARCH guest test cases --make=MAKE use specified make [$make] --python=PYTHON use specified python [$python] - --meson=MESON use specified meson [$meson] --ninja=NINJA use specified ninja [$ninja] --smbd=SMBD use specified smbd [$smbd] --with-git=GIT use specified git [$git] @@ -1090,7 +1086,6 @@ fi # Resolve PATH python="$(command -v "$python")" -explicit_python=yes # Create a Python virtual environment using our configured python. # The stdout of this script will be the location of a symlink that @@ -1102,7 +1097,6 @@ explicit_python=yes # - venv is cleared if it exists already; # - venv is allowed to use system packages; # - all setup is performed **offline**; -# - No packages are installed by default; # - pip is not installed into the venv when possible, # but ensurepip is called as a fallback when necessary. @@ -1117,58 +1111,21 @@ fi # Suppress writing compiled files python="$python -B" -has_meson() { - local python_dir=$(dirname "$python") - # PEP405: pyvenv.cfg is either adjacent to the Python executable - # or one directory above - if test -f $python_dir/pyvenv.cfg || test -f $python_dir/../pyvenv.cfg; then - # Ensure that Meson and Python come from the same virtual environment - test -x "$python_dir/meson" && - test "$(command -v meson)" -ef "$python_dir/meson" - else - has meson - fi -} -if test -z "$meson"; then - if test "$explicit_python" = no && has_meson && version_ge "$(meson --version)" 0.61.5; then - meson=meson - elif test "$git_submodules_action" != 'ignore' ; then - meson=git - elif test -e "${source_path}/meson/meson.py" ; then - meson=internal - else - if test "$explicit_python" = yes; then - error_exit "--python requires using QEMU's embedded Meson distribution, but it was not found." - else - error_exit "Meson not found. Use --meson=/path/to/meson" - fi - fi -else - # Meson uses its own Python interpreter to invoke other Python scripts, - # but the user wants to use the one they specified with --python. - # - # We do not want to override the distro Python interpreter (and sometimes - # cannot: for example in Homebrew /usr/bin/meson is a bash script), so - # just require --meson=git|internal together with --python. - if test "$explicit_python" = yes; then - case "$meson" in - git | internal) ;; - *) error_exit "--python requires using QEMU's embedded Meson distribution." ;; - esac - fi +if ! $python "${source_path}/python/scripts/mkvenv.py" ensure \ + --dir "${source_path}/python/wheels" \ + --diagnose "meson" \ + "meson>=0.61.5" ; +then + exit 1 fi -if test "$meson" = git; then - git_submodules="${git_submodules} meson" -fi +# At this point, we expect Meson to be installed and available. +# We expect mkvenv or pip to have created pyvenv/bin/meson for us. +# We ignore PATH completely here: we want to use the venv's Meson +# *exclusively*. -case "$meson" in - git | internal) - meson="$python ${source_path}/meson/meson.py" - ;; - *) meson=$(command -v "$meson") ;; -esac +meson="$(cd pyvenv/bin; pwd)/meson" # Probe for ninja From patchwork Wed May 17 17:44:48 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Paolo Bonzini X-Patchwork-Id: 1782891 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=nongnu.org (client-ip=209.51.188.17; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=) Authentication-Results: legolas.ozlabs.org; dkim=pass (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=OIpDZMpa; dkim-atps=neutral Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-ECDSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4QM1G81PXFz20dq for ; Thu, 18 May 2023 04:04:32 +1000 (AEST) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1pzLF0-0001qI-6N; Wed, 17 May 2023 13:47:10 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pzLEJ-0008UH-Hc for qemu-devel@nongnu.org; Wed, 17 May 2023 13:46:35 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pzLEI-0004KB-1P for qemu-devel@nongnu.org; Wed, 17 May 2023 13:46:27 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1684345585; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=rw+P8uHeSMURxVOwYqGlxgsOL1z8Zr5ZjIA/UkRecQ4=; b=OIpDZMpa7shV8i6/NamLoQfzipnK9rtrmMOeWehkPLtKYZc04btiYkFnGJHjhT6cY3AHIQ IBdOZ18yJYHfRtTAB5DpspMvYGrStMaGPNiHKt2Q5bb0KtieJI1JJHAFX2v0ALK/+wd71S FsBYoLxsLT+1zuCcbbepUELbS5FoFVs= Received: from mail-wm1-f69.google.com (mail-wm1-f69.google.com [209.85.128.69]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-662-n9GfwjuAMPi_fde_fTZQBQ-1; Wed, 17 May 2023 13:46:24 -0400 X-MC-Unique: n9GfwjuAMPi_fde_fTZQBQ-1 Received: by mail-wm1-f69.google.com with SMTP id 5b1f17b1804b1-3f42b226871so4969505e9.0 for ; Wed, 17 May 2023 10:46:24 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1684345583; x=1686937583; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=rw+P8uHeSMURxVOwYqGlxgsOL1z8Zr5ZjIA/UkRecQ4=; b=IkVNTl0lmoSAcT8NxePAXUROAYLuX+TN5PDUaQTMrLJEZaUPotf+aMacNISBXlUL1C DRrKsfPM04viNzGXl9AOkSNOJBYyTKfZbuPiRHS8+A8uysuTJULZeP52sICJ6nHai2Jr qNB43Z3W9hdqDcRwn/S19TabFbGKaU/lrA38V6WLAGiJmgMPoUiKGYxcGWVHLkC/wXX/ wadw0PzFaRZaDe1gPF2MhSYWFmrR7b4rYdTqyfizsFsOlj3sS706lljkhzKRPs/KeLgT dHAphSdz+OSTpm7FZaMNXwx4MzV91Jbz5WSjZc75iTAGzoWieyjEFZivrckELnxJLrK1 xWwg== X-Gm-Message-State: AC+VfDzBxi1WVLQkcqlOfI0TQiiMPoqNDHh3lH6HqOzTE+Fb1P1M4kx1 vu+xLWc+sCFTUGYgI4uV5o93e+XoOpPAh+gaU8iiktuwpUHPC4X9E3gkHH0TGofM8C39Bo3asng Z6WrOdvfqsOQJPbP6oDeZ2NlU/JZZVEEwLOWRYYhplGr3S7yADVTPZWtTqu7pzXOPP9ablP1wMy 8= X-Received: by 2002:a1c:7315:0:b0:3f1:bb10:c865 with SMTP id d21-20020a1c7315000000b003f1bb10c865mr27856168wmb.38.1684345582738; Wed, 17 May 2023 10:46:22 -0700 (PDT) X-Google-Smtp-Source: ACHHUZ7N+gHhTxbPUS3UGeXA67m1bJXsTrjlV2gpUeH/qZCzmP/r9077Wqqf83jYoqiHp438TVvlUg== X-Received: by 2002:a1c:7315:0:b0:3f1:bb10:c865 with SMTP id d21-20020a1c7315000000b003f1bb10c865mr27856157wmb.38.1684345582392; Wed, 17 May 2023 10:46:22 -0700 (PDT) Received: from [192.168.10.118] ([2001:b07:6468:f312:9af8:e5f5:7516:fa89]) by smtp.gmail.com with ESMTPSA id w19-20020a1cf613000000b003f42461ac75sm2903100wmc.12.2023.05.17.10.46.21 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 17 May 2023 10:46:21 -0700 (PDT) From: Paolo Bonzini To: qemu-devel@nongnu.org Cc: John Snow Subject: [PULL 36/68] qemu.git: drop meson git submodule Date: Wed, 17 May 2023 19:44:48 +0200 Message-Id: <20230517174520.887405-37-pbonzini@redhat.com> X-Mailer: git-send-email 2.40.1 In-Reply-To: <20230517174520.887405-1-pbonzini@redhat.com> References: <20230517174520.887405-1-pbonzini@redhat.com> MIME-Version: 1.0 Received-SPF: pass client-ip=170.10.133.124; envelope-from=pbonzini@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org From: John Snow Now that meson is installed from a vendored wheel, we don't need the git submodule anymore. Drop it. Signed-off-by: John Snow Message-Id: <20230511035435.734312-19-jsnow@redhat.com> Signed-off-by: Paolo Bonzini --- .gitmodules | 3 --- meson | 1 - 2 files changed, 4 deletions(-) delete mode 160000 meson diff --git a/.gitmodules b/.gitmodules index 6ce5bf49c588..2a3a12033c4b 100644 --- a/.gitmodules +++ b/.gitmodules @@ -49,9 +49,6 @@ [submodule "roms/qboot"] path = roms/qboot url = https://gitlab.com/qemu-project/qboot.git -[submodule "meson"] - path = meson - url = https://gitlab.com/qemu-project/meson.git [submodule "roms/vbootrom"] path = roms/vbootrom url = https://gitlab.com/qemu-project/vbootrom.git diff --git a/meson b/meson deleted file mode 160000 index 3a9b285a55b9..000000000000 --- a/meson +++ /dev/null @@ -1 +0,0 @@ -Subproject commit 3a9b285a55b91b53b2acda987192274352ecb5be From patchwork Wed May 17 17:44:49 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Paolo Bonzini X-Patchwork-Id: 1782892 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=nongnu.org (client-ip=209.51.188.17; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=) Authentication-Results: legolas.ozlabs.org; dkim=pass (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=WI1vwqAG; dkim-atps=neutral Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-ECDSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4QM1G902Lkz20dn for ; Thu, 18 May 2023 04:04:33 +1000 (AEST) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1pzLF0-00023b-KV; Wed, 17 May 2023 13:47:10 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pzLER-000054-Fi for qemu-devel@nongnu.org; Wed, 17 May 2023 13:46:37 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pzLEN-0004Ku-Nx for qemu-devel@nongnu.org; Wed, 17 May 2023 13:46:35 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1684345588; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=qoLXEPjxyhKVbWnopTIL/PIeJORoRYWKQVvdTCXP6xU=; b=WI1vwqAGgRtsION+iB6ovcDIDxt8zIEpiR5uUw/heGzcyZmzu6v9xHFg/5zv0HFi+mUgWP 7uCFD870W0opavOUKjzGN2Mvzex0EIUaJDM/Q4ZHfnIIJhitfiEPPuv+cbZCSYh1Pmyyf8 0D/p50ce3+DUrRbLrmX/NPLDRJdmo1Y= Received: from mail-wm1-f70.google.com (mail-wm1-f70.google.com [209.85.128.70]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-320-T6w6aIZ8PdCOOwaRjbdKVg-1; Wed, 17 May 2023 13:46:27 -0400 X-MC-Unique: T6w6aIZ8PdCOOwaRjbdKVg-1 Received: by mail-wm1-f70.google.com with SMTP id 5b1f17b1804b1-3f42ceb4a4fso4346535e9.0 for ; Wed, 17 May 2023 10:46:25 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1684345584; x=1686937584; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=qoLXEPjxyhKVbWnopTIL/PIeJORoRYWKQVvdTCXP6xU=; b=VIY4lWc+eCaZ512AxURYqzqf//0/bkaZq/qiRjeSZxNTwgmdEV0/h7qCUM7pGWIPZz 3cIsOH9SKhBeXo7nO2Xof7vSFgINDog2Pl2gs7W5NF5ubVvOYaZ1+R6EvJGWQ9nEiX1w z7DuJQ66YpSENOT1nDZ0I1fQfeunpN/LkmFNu4UTyNht+jVbkdceTsP/u5Al0Yd8jdZu MxIQG0Bt8itRJfATUGo2d6QQuu4skVYvI3PsCWJhWb/+UfrGjR/aV0yPy4Wt72UZNCWH v7s5dPchVd3BX/0zw6ObORaeRgYo/XDUC6JsuMS+XuNmrBGpY/8uMGWW4yI+KcykMDTG q1jA== X-Gm-Message-State: AC+VfDzcdKUKbX0Vn8kB9mTN0TCMZBA8pSjx3HDopu6VndRgJ8H7PY4m 7gO3L+YADK9ps6sCQMQ77flrvPBx2+QAHFnkuK4+qZXFLGxLf5AcvsC/m0zO3U87awSkwXzDVVs JCJrUj3e5roD4CzmHY5fo1ZlFbipmIxyRJjQZPTc2oXnaewq0KLs7ftCxpf4eQkRp5HAiNR8p+L Q= X-Received: by 2002:a05:600c:3655:b0:3f4:2cc7:aac5 with SMTP id y21-20020a05600c365500b003f42cc7aac5mr19615494wmq.9.1684345584341; Wed, 17 May 2023 10:46:24 -0700 (PDT) X-Google-Smtp-Source: ACHHUZ4iNmIB+acFtq2E/j15sjhuM5D1BNxlnGKp82JvzYPikiCMMy0uHomAiVYusvJO16hYOm5SwA== X-Received: by 2002:a05:600c:3655:b0:3f4:2cc7:aac5 with SMTP id y21-20020a05600c365500b003f42cc7aac5mr19615484wmq.9.1684345583912; Wed, 17 May 2023 10:46:23 -0700 (PDT) Received: from [192.168.10.118] ([2001:b07:6468:f312:9af8:e5f5:7516:fa89]) by smtp.gmail.com with ESMTPSA id k20-20020a7bc414000000b003f18b942338sm2892765wmi.3.2023.05.17.10.46.22 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 17 May 2023 10:46:23 -0700 (PDT) From: Paolo Bonzini To: qemu-devel@nongnu.org Cc: John Snow Subject: [PULL 37/68] tests: Use configure-provided pyvenv for tests Date: Wed, 17 May 2023 19:44:49 +0200 Message-Id: <20230517174520.887405-38-pbonzini@redhat.com> X-Mailer: git-send-email 2.40.1 In-Reply-To: <20230517174520.887405-1-pbonzini@redhat.com> References: <20230517174520.887405-1-pbonzini@redhat.com> MIME-Version: 1.0 Received-SPF: pass client-ip=170.10.133.124; envelope-from=pbonzini@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org From: John Snow This patch changes how the avocado tests are provided, ever so slightly. Instead of creating a new testing venv, use the configure-provided 'pyvenv' instead, and install optional packages into that. Signed-off-by: John Snow Message-Id: <20230511035435.734312-20-jsnow@redhat.com> Signed-off-by: Paolo Bonzini --- .gitlab-ci.d/buildtest.yml | 6 +++--- docs/devel/acpi-bits.rst | 6 +++--- docs/devel/testing.rst | 14 +++++++------- scripts/ci/org.centos/stream/8/x86_64/test-avocado | 4 ++-- scripts/device-crash-test | 2 +- tests/Makefile.include | 10 +++++----- tests/requirements.txt | 7 +++++-- 7 files changed, 26 insertions(+), 23 deletions(-) diff --git a/.gitlab-ci.d/buildtest.yml b/.gitlab-ci.d/buildtest.yml index bb3650a51c09..307cba1aab6d 100644 --- a/.gitlab-ci.d/buildtest.yml +++ b/.gitlab-ci.d/buildtest.yml @@ -103,7 +103,7 @@ crash-test-debian: script: - cd build - make NINJA=":" check-venv - - tests/venv/bin/python3 scripts/device-crash-test -q --tcg-only ./qemu-system-i386 + - pyvenv/bin/python3 scripts/device-crash-test -q --tcg-only ./qemu-system-i386 build-system-fedora: extends: @@ -146,8 +146,8 @@ crash-test-fedora: script: - cd build - make NINJA=":" check-venv - - tests/venv/bin/python3 scripts/device-crash-test -q ./qemu-system-ppc - - tests/venv/bin/python3 scripts/device-crash-test -q ./qemu-system-riscv32 + - pyvenv/bin/python3 scripts/device-crash-test -q ./qemu-system-ppc + - pyvenv/bin/python3 scripts/device-crash-test -q ./qemu-system-riscv32 build-system-centos: extends: diff --git a/docs/devel/acpi-bits.rst b/docs/devel/acpi-bits.rst index 22e2580200cd..9677b0098f4b 100644 --- a/docs/devel/acpi-bits.rst +++ b/docs/devel/acpi-bits.rst @@ -61,19 +61,19 @@ Under ``tests/avocado/`` as the root we have: :: $ make check-venv (needed only the first time to create the venv) - $ ./tests/venv/bin/avocado run -t acpi tests/avocado + $ ./pyvenv/bin/avocado run -t acpi tests/avocado The above will run all acpi avocado tests including this one. In order to run the individual tests, perform the following: :: - $ ./tests/venv/bin/avocado run tests/avocado/acpi-bits.py --tap - + $ ./pyvenv/bin/avocado run tests/avocado/acpi-bits.py --tap - The above will produce output in tap format. You can omit "--tap -" in the end and it will produce output like the following: :: - $ ./tests/venv/bin/avocado run tests/avocado/acpi-bits.py + $ ./pyvenv/bin/avocado run tests/avocado/acpi-bits.py Fetching asset from tests/avocado/acpi-bits.py:AcpiBitsTest.test_acpi_smbios_bits JOB ID : eab225724da7b64c012c65705dc2fa14ab1defef JOB LOG : /home/anisinha/avocado/job-results/job-2022-10-10T17.58-eab2257/job.log diff --git a/docs/devel/testing.rst b/docs/devel/testing.rst index 4071e727107a..50664d9eb9d5 100644 --- a/docs/devel/testing.rst +++ b/docs/devel/testing.rst @@ -882,9 +882,9 @@ You can run the avocado tests simply by executing: make check-avocado -This involves the automatic creation of Python virtual environment -within the build tree (at ``tests/venv``) which will have all the -right dependencies, and will save tests results also within the +This involves the automatic installation, from PyPI, of all the +necessary avocado-framework dependencies into the QEMU venv within the +build tree (at ``./pyvenv``). Test results are also saved within the build tree (at ``tests/results``). Note: the build environment must be using a Python 3 stack, and have @@ -941,7 +941,7 @@ may be invoked by running: .. code:: - tests/venv/bin/avocado run $OPTION1 $OPTION2 tests/avocado/ + pyvenv/bin/avocado run $OPTION1 $OPTION2 tests/avocado/ Note that if ``make check-avocado`` was not executed before, it is possible to create the Python virtual environment with the dependencies @@ -956,20 +956,20 @@ a test file. To run tests from a single file within the build tree, use: .. code:: - tests/venv/bin/avocado run tests/avocado/$TESTFILE + pyvenv/bin/avocado run tests/avocado/$TESTFILE To run a single test within a test file, use: .. code:: - tests/venv/bin/avocado run tests/avocado/$TESTFILE:$TESTCLASS.$TESTNAME + pyvenv/bin/avocado run tests/avocado/$TESTFILE:$TESTCLASS.$TESTNAME Valid test names are visible in the output from any previous execution of Avocado or ``make check-avocado``, and can also be queried using: .. code:: - tests/venv/bin/avocado list tests/avocado + pyvenv/bin/avocado list tests/avocado Manual Installation ~~~~~~~~~~~~~~~~~~~ diff --git a/scripts/ci/org.centos/stream/8/x86_64/test-avocado b/scripts/ci/org.centos/stream/8/x86_64/test-avocado index d2c0e5fb4cc8..7bb5b317b64d 100755 --- a/scripts/ci/org.centos/stream/8/x86_64/test-avocado +++ b/scripts/ci/org.centos/stream/8/x86_64/test-avocado @@ -4,7 +4,7 @@ # KVM and x86_64, or tests that are generic enough to be valid for all # targets. Such a test list can be generated with: # -# ./tests/venv/bin/avocado list --filter-by-tags-include-empty \ +# ./pyvenv/bin/avocado list --filter-by-tags-include-empty \ # --filter-by-tags-include-empty-key -t accel:kvm,arch:x86_64 \ # tests/avocado/ # @@ -22,7 +22,7 @@ # - tests/avocado/virtio_check_params.py:VirtioMaxSegSettingsCheck.test_machine_types # make get-vm-images -./tests/venv/bin/avocado run \ +./pyvenv/bin/avocado run \ --job-results-dir=tests/results/ \ tests/avocado/boot_linux.py:BootLinuxX8664.test_pc_i440fx_kvm \ tests/avocado/boot_linux.py:BootLinuxX8664.test_pc_q35_kvm \ diff --git a/scripts/device-crash-test b/scripts/device-crash-test index b74d887331db..353aa575d7b7 100755 --- a/scripts/device-crash-test +++ b/scripts/device-crash-test @@ -43,7 +43,7 @@ except ModuleNotFoundError as exc: print(f"Module '{exc.name}' not found.") print(" Try 'make check-venv' from your build directory,") print(" and then one way to run this script is like so:") - print(f' > $builddir/tests/venv/bin/python3 "{path}"') + print(f' > $builddir/pyvenv/bin/python3 "{path}"') sys.exit(1) logger = logging.getLogger('device-crash-test') diff --git a/tests/Makefile.include b/tests/Makefile.include index a4de0ad5a252..5b838ec438b0 100644 --- a/tests/Makefile.include +++ b/tests/Makefile.include @@ -89,7 +89,8 @@ distclean-tcg: $(DISTCLEAN_TCG_TARGET_RULES) # Build up our target list from the filtered list of ninja targets TARGETS=$(patsubst libqemu-%.fa, %, $(filter libqemu-%.fa, $(ninja-targets))) -TESTS_VENV_DIR=$(BUILD_DIR)/tests/venv +TESTS_VENV_DIR=$(BUILD_DIR)/pyvenv +TESTS_VENV_TOKEN=$(BUILD_DIR)/pyvenv/tests.group TESTS_VENV_REQ=$(SRC_PATH)/tests/requirements.txt TESTS_RESULTS_DIR=$(BUILD_DIR)/tests/results TESTS_PYTHON=$(TESTS_VENV_DIR)/bin/python3 @@ -111,8 +112,7 @@ quiet-venv-pip = $(quiet-@)$(call quiet-command-run, \ $(TESTS_PYTHON) -m pip -q --disable-pip-version-check $1, \ "VENVPIP","$1") -$(TESTS_VENV_DIR): $(TESTS_VENV_REQ) - $(call quiet-command, $(PYTHON) -m venv $@, VENV, $@) +$(TESTS_VENV_TOKEN): $(TESTS_VENV_REQ) $(call quiet-venv-pip,install -e "$(SRC_PATH)/python/") $(call quiet-venv-pip,install -r $(TESTS_VENV_REQ)) $(call quiet-command, touch $@) @@ -121,7 +121,7 @@ $(TESTS_RESULTS_DIR): $(call quiet-command, mkdir -p $@, \ MKDIR, $@) -check-venv: $(TESTS_VENV_DIR) +check-venv: $(TESTS_VENV_TOKEN) FEDORA_31_ARCHES_TARGETS=$(patsubst %-softmmu,%, $(filter %-softmmu,$(TARGETS))) FEDORA_31_ARCHES_CANDIDATES=$(patsubst ppc64,ppc64le,$(FEDORA_31_ARCHES_TARGETS)) @@ -167,7 +167,7 @@ check: check-build: run-ninja check-clean: - rm -rf $(TESTS_VENV_DIR) $(TESTS_RESULTS_DIR) + rm -rf $(TESTS_RESULTS_DIR) clean: check-clean clean-tcg distclean: distclean-tcg diff --git a/tests/requirements.txt b/tests/requirements.txt index a6f73da6811a..0e008b9aec35 100644 --- a/tests/requirements.txt +++ b/tests/requirements.txt @@ -1,6 +1,9 @@ # Add Python module requirements, one per line, to be installed -# in the tests/venv Python virtual environment. For more info, +# in the qemu build_dir/pyvenv Python virtual environment. For more info, # refer to: https://pip.pypa.io/en/stable/user_guide/#id1 -# Note that qemu.git/python/ is always implicitly installed. +# +# Note that qemu.git/python/ is implicitly installed to this venv when +# 'make check-venv' is run, and will persist until configure is run +# again. avocado-framework==101.0 pycdlib==1.11.0 From patchwork Wed May 17 17:44:50 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Paolo Bonzini X-Patchwork-Id: 1782855 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=nongnu.org (client-ip=209.51.188.17; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=) Authentication-Results: legolas.ozlabs.org; dkim=pass (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=iYlHgV2w; dkim-atps=neutral Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-ECDSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4QM12W3xtLz20KF for ; Thu, 18 May 2023 03:54:27 +1000 (AEST) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1pzLEY-0000LQ-MZ; Wed, 17 May 2023 13:46:42 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pzLER-00004z-8P for qemu-devel@nongnu.org; Wed, 17 May 2023 13:46:36 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pzLEN-0004Kz-Ns for qemu-devel@nongnu.org; Wed, 17 May 2023 13:46:34 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1684345588; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=UPbo2+VzDqysnyV4UzfBcKr3OVEdwOudnHphko4hQ4w=; b=iYlHgV2wlrQHGUbbiZI6sES6NgQV+PRUVzNmiKNkrjIYjExiqMzOZUlCeFAg6rkRR6CHdU xYpBG/II0tczQpKmL3X8NH375luCHe5/d8coA3n/nhzrWT7ML8pr6iDJ+JCmYE/yOEhhzH spjmFbqF5OfAxfMSrBnQHZXdRcesbj4= Received: from mail-wr1-f69.google.com (mail-wr1-f69.google.com [209.85.221.69]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-376-mE2KU8lZOkSqmoTvV8gV6g-1; Wed, 17 May 2023 13:46:27 -0400 X-MC-Unique: mE2KU8lZOkSqmoTvV8gV6g-1 Received: by mail-wr1-f69.google.com with SMTP id ffacd0b85a97d-3079d15b51dso165825f8f.1 for ; Wed, 17 May 2023 10:46:27 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1684345586; x=1686937586; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=UPbo2+VzDqysnyV4UzfBcKr3OVEdwOudnHphko4hQ4w=; b=TJfaTXckWYaH/Tq0JX/le7o8BGkh19VCRldsTcrSH3tbLKFWZr9n+MJNilzbxQTuNw t/TJeKeZoSMUY1z1+cchR0FPlfTzAJU7nHzEEFUKlgnoytNWCP3Gx3wmi5EC4iPkyXtW wPp1s0XUdGVqE3gv7bnpi0RvFIJQRnj6zvnunjq1b9EhO2gcFQ62+Ha1z++5d+qvTg8y HjixNssdBRE/whzTLmf6Y3tUaFZikBYh2CHKfuowGKbg4IyHy+PB9fW4RzLZ1qTjWDZc x11ad9vioAAjm3jw3Js1otmxnwxp/uZS30JwzK1PM/u1zescJx4MRt7LJjji4E9rxnRQ KByA== X-Gm-Message-State: AC+VfDzVrFn4bLuSH847WzH3q5MJc1FNiihDqQHH740njhbq/y4nbILB Tu/X0PJ/3xCojMVIFnHeGZY/7GaOz5rZmA0YMqD6gBFGXdDJYg+wxGR5L+AUmftyhSSnLQjjvFQ 6MGyd4lMjT8WvTwmanpRsT/QduUx9fqkshvs+Bj2LAa9B4ISc16rTQ3w6fpxM05w7HKHm0tRFh8 c= X-Received: by 2002:adf:eb4b:0:b0:307:9693:efe1 with SMTP id u11-20020adfeb4b000000b003079693efe1mr1243636wrn.18.1684345586123; Wed, 17 May 2023 10:46:26 -0700 (PDT) X-Google-Smtp-Source: ACHHUZ7Lx2V6JfmAjUsaLFTEGYM6pqqi9FcmhChwEelud65zaDcrxzrpxnhnyMQtMgxFubFCig3JQw== X-Received: by 2002:adf:eb4b:0:b0:307:9693:efe1 with SMTP id u11-20020adfeb4b000000b003079693efe1mr1243618wrn.18.1684345585712; Wed, 17 May 2023 10:46:25 -0700 (PDT) Received: from [192.168.10.118] ([2001:b07:6468:f312:9af8:e5f5:7516:fa89]) by smtp.gmail.com with ESMTPSA id i4-20020a5d55c4000000b002ca864b807csm3737423wrw.0.2023.05.17.10.46.24 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 17 May 2023 10:46:25 -0700 (PDT) From: Paolo Bonzini To: qemu-devel@nongnu.org Cc: John Snow Subject: [PULL 38/68] configure: move --enable-docs and --disable-docs back to configure Date: Wed, 17 May 2023 19:44:50 +0200 Message-Id: <20230517174520.887405-39-pbonzini@redhat.com> X-Mailer: git-send-email 2.40.1 In-Reply-To: <20230517174520.887405-1-pbonzini@redhat.com> References: <20230517174520.887405-1-pbonzini@redhat.com> MIME-Version: 1.0 Received-SPF: pass client-ip=170.10.133.124; envelope-from=pbonzini@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org From: John Snow Move this option back from meson into configure for the purposes of using the configuration value to bootstrap Sphinx in different ways based on this value. Signed-off-by: John Snow Message-Id: <20230511035435.734312-21-jsnow@redhat.com> Signed-off-by: Paolo Bonzini --- configure | 6 ++++++ 1 file changed, 6 insertions(+) diff --git a/configure b/configure index 4519c4d412f8..f8308926bbc6 100755 --- a/configure +++ b/configure @@ -279,6 +279,7 @@ debug_tcg="no" sanitizers="no" tsan="no" fortify_source="yes" +docs="auto" EXESUF="" modules="no" prefix="/usr/local" @@ -750,6 +751,10 @@ for opt do ;; --disable-debug-info) meson_option_add -Ddebug=false ;; + --enable-docs) docs=enabled + ;; + --disable-docs) docs=disabled + ;; --enable-modules) modules="yes" ;; @@ -2615,6 +2620,7 @@ if test "$skip_meson" = no; then # QEMU options test "$cfi" != false && meson_option_add "-Dcfi=$cfi" + test "$docs" != auto && meson_option_add "-Ddocs=$docs" test "$fdt" != auto && meson_option_add "-Dfdt=$fdt" test -n "${LIB_FUZZING_ENGINE+xxx}" && meson_option_add "-Dfuzzing_engine=$LIB_FUZZING_ENGINE" test "$qemu_suffix" != qemu && meson_option_add "-Dqemu_suffix=$qemu_suffix" From patchwork Wed May 17 17:44:51 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Paolo Bonzini X-Patchwork-Id: 1782853 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=nongnu.org (client-ip=209.51.188.17; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=) Authentication-Results: legolas.ozlabs.org; dkim=pass (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=G4cFkgQ+; dkim-atps=neutral Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-ECDSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4QM11r4Mfqz20KF for ; Thu, 18 May 2023 03:53:52 +1000 (AEST) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1pzLGF-0003nS-16; Wed, 17 May 2023 13:48:27 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pzLER-000050-9J for qemu-devel@nongnu.org; Wed, 17 May 2023 13:46:36 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.129.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pzLEN-0004LB-OE for qemu-devel@nongnu.org; Wed, 17 May 2023 13:46:35 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1684345590; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=04MmlIKoOQC2vxWjCqSmC8U6PoC6qnjYxWljRrYw6JI=; b=G4cFkgQ+ZApvUocuz829tjuYA2QWl7L13WV8IAGACOtTK6BAiMtQscJ2hTRQiDF1HiEZd5 ue8cFTRi0Q7IrReiEJA4SMbEsdSJzUY9ibhP7LBlt+Lj1U0OG11zNWTsrZPYJCn+EHlIXy EWNeHbwAYmWsrq0xv4LRjRAnH60u8R8= Received: from mail-wm1-f72.google.com (mail-wm1-f72.google.com [209.85.128.72]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-561-mtQCqQONMOWv7nmpgY3hiQ-1; Wed, 17 May 2023 13:46:29 -0400 X-MC-Unique: mtQCqQONMOWv7nmpgY3hiQ-1 Received: by mail-wm1-f72.google.com with SMTP id 5b1f17b1804b1-3f420742d40so4452355e9.2 for ; Wed, 17 May 2023 10:46:29 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1684345587; x=1686937587; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=04MmlIKoOQC2vxWjCqSmC8U6PoC6qnjYxWljRrYw6JI=; b=GGv9CmfZljTavOYdHc413GApgps3rcL/DYBo/U4xvkxUfmY/VY6o8tWtGRGoE/KOPf GmZn+XH48EOTwZQqkiNqiCA/nnoZhaVtL1uj/wtqqPTPJ+h3PmU2SJYouHcg7sC0wde3 RiBGRBrLNPmYvy3maW6AMVgBCirvmEz6/VtxaoaBIzEXedrL8KFv3bL3mq8v8OcLiuNY dCC0HNVyanfyyUAP5FklgbfesDyBZMrRYvLwpVYAgsccU3t23xWP/pc6b2NX05kRvyAt UGwZq5n92SqlXv+O0WUvaChuImQk47lH2s/49Fiz1XKxZGe5B7v6G1suXBFQXq+AVN8L Ka8w== X-Gm-Message-State: AC+VfDzUPbEv58iUKKyfHwPVdMaoHswznqMxOviiNu+zqWmj7QWF1Lv7 7A6iOuK90PRdp5LwggbUMTaoZI47LXNOBsCWMsk27H26INFRWmALtOVeoY3aTS5k5Rw0YuDfmr1 T1YvJdOc68oOHsn6qrQFT0t04OBRjokVCj4n2bH4oHMp1HdgCY1fx+T5pJUdl7BOtRfx5lJ55JE 0= X-Received: by 2002:a05:600c:2190:b0:3f4:2b13:f0fb with SMTP id e16-20020a05600c219000b003f42b13f0fbmr22790339wme.13.1684345587600; Wed, 17 May 2023 10:46:27 -0700 (PDT) X-Google-Smtp-Source: ACHHUZ5IrYAKrofgknQzYm/T01ERVltDYL8Ph/Whs0cw9dBbFysDo33Czss0h031W0oPbuC+MQMTig== X-Received: by 2002:a05:600c:2190:b0:3f4:2b13:f0fb with SMTP id e16-20020a05600c219000b003f42b13f0fbmr22790326wme.13.1684345587261; Wed, 17 May 2023 10:46:27 -0700 (PDT) Received: from [192.168.10.118] ([2001:b07:6468:f312:9af8:e5f5:7516:fa89]) by smtp.gmail.com with ESMTPSA id l8-20020a1c7908000000b003f506e6ff83sm2873629wme.22.2023.05.17.10.46.26 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 17 May 2023 10:46:26 -0700 (PDT) From: Paolo Bonzini To: qemu-devel@nongnu.org Cc: John Snow Subject: [PULL 39/68] configure: bootstrap sphinx with mkvenv Date: Wed, 17 May 2023 19:44:51 +0200 Message-Id: <20230517174520.887405-40-pbonzini@redhat.com> X-Mailer: git-send-email 2.40.1 In-Reply-To: <20230517174520.887405-1-pbonzini@redhat.com> References: <20230517174520.887405-1-pbonzini@redhat.com> MIME-Version: 1.0 Received-SPF: pass client-ip=170.10.129.124; envelope-from=pbonzini@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org From: John Snow When docs are explicitly requested, require Sphinx>=1.6.0. When docs are explicitly disabled, don't bother to check for Sphinx at all. If docs are set to "auto", attempt to locate Sphinx, but continue onward if it wasn't located. Signed-off-by: Paolo Bonzini Signed-off-by: John Snow Message-Id: <20230511035435.734312-22-jsnow@redhat.com> Signed-off-by: Paolo Bonzini --- configure | 21 +++++++++++++++++++-- docs/conf.py | 9 --------- docs/meson.build | 2 +- meson_options.txt | 2 -- scripts/meson-buildoptions.sh | 3 --- 5 files changed, 20 insertions(+), 17 deletions(-) diff --git a/configure b/configure index f8308926bbc6..411140a306ff 100755 --- a/configure +++ b/configure @@ -1115,9 +1115,9 @@ fi # Suppress writing compiled files python="$python -B" +mkvenv="$python ${source_path}/python/scripts/mkvenv.py" - -if ! $python "${source_path}/python/scripts/mkvenv.py" ensure \ +if ! $mkvenv ensure \ --dir "${source_path}/python/wheels" \ --diagnose "meson" \ "meson>=0.61.5" ; @@ -1132,6 +1132,23 @@ fi meson="$(cd pyvenv/bin; pwd)/meson" +# Conditionally ensure Sphinx is installed. + +if test "$docs" != "disabled" ; then + if ! $mkvenv ensure \ + --diagnose "sphinx-build" \ + "sphinx>=1.6.0" sphinx-rtd-theme; + then + if test "$docs" = "enabled" ; then + exit 1 + fi + echo "Sphinx not found/usable, disabling docs." + docs=disabled + else + docs=enabled + fi +fi + # Probe for ninja if test -z "$ninja"; then diff --git a/docs/conf.py b/docs/conf.py index 00767b0e24fb..c687ff266301 100644 --- a/docs/conf.py +++ b/docs/conf.py @@ -32,15 +32,6 @@ from distutils.version import LooseVersion from sphinx.errors import ConfigError -# Make Sphinx fail cleanly if using an old Python, rather than obscurely -# failing because some code in one of our extensions doesn't work there. -# In newer versions of Sphinx this will display nicely; in older versions -# Sphinx will also produce a Python backtrace but at least the information -# gets printed... -if sys.version_info < (3,6): - raise ConfigError( - "QEMU requires a Sphinx that uses Python 3.6 or better\n") - # The per-manual conf.py will set qemu_docdir for a single-manual build; # otherwise set it here if this is an entire-manual-set build. # This is always the absolute path of the docs/ directory in the source tree. diff --git a/docs/meson.build b/docs/meson.build index 6d0986579e17..9040f860ae1a 100644 --- a/docs/meson.build +++ b/docs/meson.build @@ -1,4 +1,4 @@ -sphinx_build = find_program(get_option('sphinx_build'), +sphinx_build = find_program(fs.parent(python.full_path()) / 'sphinx-build', required: get_option('docs')) # Check if tools are available to build documentation. diff --git a/meson_options.txt b/meson_options.txt index d8330a1f71e7..a350520f6abf 100644 --- a/meson_options.txt +++ b/meson_options.txt @@ -12,8 +12,6 @@ option('pkgversion', type : 'string', value : '', description: 'use specified string as sub-version of the package') option('smbd', type : 'string', value : '', description: 'Path to smbd for slirp networking') -option('sphinx_build', type : 'string', value : 'sphinx-build', - description: 'Use specified sphinx-build for building document') option('iasl', type : 'string', value : '', description: 'Path to ACPI disassembler') option('tls_priority', type : 'string', value : 'NORMAL', diff --git a/scripts/meson-buildoptions.sh b/scripts/meson-buildoptions.sh index 2805d1c14540..fedb93ada69e 100644 --- a/scripts/meson-buildoptions.sh +++ b/scripts/meson-buildoptions.sh @@ -57,8 +57,6 @@ meson_options_help() { printf "%s\n" ' --localedir=VALUE Locale data directory [share/locale]' printf "%s\n" ' --localstatedir=VALUE Localstate data directory [/var/local]' printf "%s\n" ' --mandir=VALUE Manual page directory [share/man]' - printf "%s\n" ' --sphinx-build=VALUE Use specified sphinx-build for building document' - printf "%s\n" ' [sphinx-build]' printf "%s\n" ' --sysconfdir=VALUE Sysconf data directory [etc]' printf "%s\n" ' --tls-priority=VALUE Default TLS protocol/cipher priority string' printf "%s\n" ' [NORMAL]' @@ -425,7 +423,6 @@ _meson_option_parse() { --disable-sndio) printf "%s" -Dsndio=disabled ;; --enable-sparse) printf "%s" -Dsparse=enabled ;; --disable-sparse) printf "%s" -Dsparse=disabled ;; - --sphinx-build=*) quote_sh "-Dsphinx_build=$2" ;; --enable-spice) printf "%s" -Dspice=enabled ;; --disable-spice) printf "%s" -Dspice=disabled ;; --enable-spice-protocol) printf "%s" -Dspice_protocol=enabled ;; From patchwork Wed May 17 17:44:52 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Paolo Bonzini X-Patchwork-Id: 1782842 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=nongnu.org (client-ip=209.51.188.17; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=) Authentication-Results: legolas.ozlabs.org; dkim=pass (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=HrJWnAB7; dkim-atps=neutral Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-ECDSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4QM0wr080Cz20dr for ; Thu, 18 May 2023 03:49:32 +1000 (AEST) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1pzLEZ-0000Om-Ba; Wed, 17 May 2023 13:46:43 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pzLET-00005q-Gx for qemu-devel@nongnu.org; Wed, 17 May 2023 13:46:40 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pzLEP-0004LJ-EC for qemu-devel@nongnu.org; Wed, 17 May 2023 13:46:35 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1684345591; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=Ra6PhvxA9OTVLts8tx91DcK1aYd+H54sTUC//9X5GXQ=; b=HrJWnAB7NvMtDjSNBnLY72OpE2/UBWwKkHDGsnza+MmZTogW7c6RzKBs4ZQ+9QjDRFNqfj 1BoOkRpgS3yZgXi1FvKGvRYZDqsmN7WxfXo6EwcHdDQ5f8DaucdbgZgXV9wTgbgFA6mvx0 BAVCzVDgZFxekzISQCCrsGAheLztTn4= Received: from mail-wr1-f70.google.com (mail-wr1-f70.google.com [209.85.221.70]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-474-vjoYMZynNlKFVjyQqs2Q4Q-1; Wed, 17 May 2023 13:46:30 -0400 X-MC-Unique: vjoYMZynNlKFVjyQqs2Q4Q-1 Received: by mail-wr1-f70.google.com with SMTP id ffacd0b85a97d-3079d15b51dso165837f8f.1 for ; Wed, 17 May 2023 10:46:30 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1684345589; x=1686937589; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=Ra6PhvxA9OTVLts8tx91DcK1aYd+H54sTUC//9X5GXQ=; b=T+wJwUKSSjUwJFcS0g40qfzPuweEHJXV9hBt/5P3owvzQuT7OU9uzQ+NPX5Hik3Pb/ tlGDoBUqiiZz5ku9x2JHBA4n0lQkFnYYuDAVQez/x9DYMxBt/qVBXRu785oP+lrUywlo vNKIriogJJAHHr3tZwHajih38DcFzeY1KdjSEIjUskVzl5gLoPet1cuO2dvyQ6TfIEiO RBM3SB0NP1yGKG61DHsQsK4f9QroWc8uxIE/ESZMMH4rV0tMEAgVZz194V8Jt/kNEnco FzofAzYGiA6eC8rGTkxA43xT1+Bnr7iu/yia/88qzQHykHWWnP28O2QCtYwP6yB3Jo3b JZng== X-Gm-Message-State: AC+VfDy2yU+N56dQvBdapSmvAuta0yME5ab5TlNmfyVPWNeWXqpXjoe0 2to+Pw4PcT6Hgg8lEmbwFpuka5CQ6KEdSzYTNQTuBZbW4bwGRpknXrm3GKx/VlVKCjt+2WXosV2 KVrMfnokPGUCSbNYLenzeFGzPRc/tNqahH/2Pr26bvKxBuomZ4pE91Qb4zoqYhAAQaEG1YmHdC+ 8= X-Received: by 2002:a5d:43c8:0:b0:306:c524:76cd with SMTP id v8-20020a5d43c8000000b00306c52476cdmr1174607wrr.27.1684345589079; Wed, 17 May 2023 10:46:29 -0700 (PDT) X-Google-Smtp-Source: ACHHUZ7LB8YmiJJmZzKDw0wpOM3n23qdDkUb/s55ddfCXYmDF4CT+ofzKH9ad+fjJw/N4PIHF2j7VA== X-Received: by 2002:a5d:43c8:0:b0:306:c524:76cd with SMTP id v8-20020a5d43c8000000b00306c52476cdmr1174596wrr.27.1684345588766; Wed, 17 May 2023 10:46:28 -0700 (PDT) Received: from [192.168.10.118] ([2001:b07:6468:f312:9af8:e5f5:7516:fa89]) by smtp.gmail.com with ESMTPSA id q15-20020adff50f000000b0030903d44dbcsm3350169wro.33.2023.05.17.10.46.27 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 17 May 2023 10:46:28 -0700 (PDT) From: Paolo Bonzini To: qemu-devel@nongnu.org Cc: John Snow Subject: [PULL 40/68] configure: add --enable-pypi and --disable-pypi Date: Wed, 17 May 2023 19:44:52 +0200 Message-Id: <20230517174520.887405-41-pbonzini@redhat.com> X-Mailer: git-send-email 2.40.1 In-Reply-To: <20230517174520.887405-1-pbonzini@redhat.com> References: <20230517174520.887405-1-pbonzini@redhat.com> MIME-Version: 1.0 Received-SPF: pass client-ip=170.10.133.124; envelope-from=pbonzini@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org From: John Snow In the event that there's no vendored source present and no sufficient version of $package can be found, we will attempt to connect to PyPI to install the package if '--disable-pypi' was not passed. This means that PyPI access is "enabled by default", but there are some subtleties that make this action occur much less frequently than you might imagine: (1) While --enable-pypi is the default, vendored source will always be preferred when found, making PyPI a fallback. This should ensure that configure-time venv building "just works" for almost everyone in almost every circumstance. (2) Because meson source is, at time of writing, vendored directly into qemu.git, PyPI will never be used for sourcing meson. (3) Because Sphinx is an optional dependency, if docs are set to "auto", PyPI will not be used to obtain Sphinx source as a fallback and instead docs will be disabled. If PyPI sourcing of sphinx is desired, --enable-docs should be passed to force the lookup. I chose this as the default behavior to avoid adding new internet lookups to a "default" invocation of configure. Signed-off-by: John Snow Message-Id: <20230511035435.734312-23-jsnow@redhat.com> Signed-off-by: Paolo Bonzini --- configure | 21 ++++++++++++++++++++- 1 file changed, 20 insertions(+), 1 deletion(-) diff --git a/configure b/configure index 411140a306ff..b92da9e6605b 100755 --- a/configure +++ b/configure @@ -623,6 +623,7 @@ check_py_version() { } python= +pypi="enabled" first_python= if test -z "${PYTHON}"; then # A bare 'python' is traditionally python 2.x, but some distros @@ -888,6 +889,10 @@ for opt do --with-git-submodules=*) git_submodules_action="$optarg" ;; + --disable-pypi) pypi="disabled" + ;; + --enable-pypi) pypi="enabled" + ;; --enable-plugins) if test "$mingw32" = "yes"; then error_exit "TCG plugins not currently supported on Windows platforms" else @@ -1101,7 +1106,9 @@ python="$(command -v "$python")" # Defaults assumed for now: # - venv is cleared if it exists already; # - venv is allowed to use system packages; -# - all setup is performed **offline**; +# - all setup can be performed offline; +# - missing packages may be fetched from PyPI, +# unless --disable-pypi is passed. # - pip is not installed into the venv when possible, # but ensurepip is called as a fallback when necessary. @@ -1117,7 +1124,13 @@ fi python="$python -B" mkvenv="$python ${source_path}/python/scripts/mkvenv.py" +mkvenv_flags="" +if test "$pypi" = "enabled" ; then + mkvenv_flags="--online" +fi + if ! $mkvenv ensure \ + $mkvenv_flags \ --dir "${source_path}/python/wheels" \ --diagnose "meson" \ "meson>=0.61.5" ; @@ -1134,8 +1147,14 @@ meson="$(cd pyvenv/bin; pwd)/meson" # Conditionally ensure Sphinx is installed. +mkvenv_flags="" +if test "$pypi" = "enabled" -a "$docs" = "enabled" ; then + mkvenv_flags="--online" +fi + if test "$docs" != "disabled" ; then if ! $mkvenv ensure \ + $mkvenv_flags \ --diagnose "sphinx-build" \ "sphinx>=1.6.0" sphinx-rtd-theme; then From patchwork Wed May 17 17:44:53 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Paolo Bonzini X-Patchwork-Id: 1782881 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=nongnu.org (client-ip=209.51.188.17; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=) Authentication-Results: legolas.ozlabs.org; dkim=pass (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=S5sidhsY; dkim-atps=neutral Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-ECDSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4QM1DY1KN9z20dn for ; Thu, 18 May 2023 04:03:09 +1000 (AEST) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1pzLFi-0003AO-Bw; Wed, 17 May 2023 13:48:05 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pzLET-00005r-HM for qemu-devel@nongnu.org; Wed, 17 May 2023 13:46:40 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.129.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pzLEQ-0004Lg-OP for qemu-devel@nongnu.org; Wed, 17 May 2023 13:46:36 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1684345594; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=VUI5gP+VGMEuKYZHL551pN3vhWgbV23BFYT1rHteqTM=; b=S5sidhsYN6WgkeOq/+70xDnhjZrTSc8NDKiimd8DM51DnunCbPCLM1Q7dZm4zC6M8yHoE1 oc4I59x5E9H0XLWfQRjM/CjF2ufj5i6n8gsCjvQsrpkj6cdIYFhEkB8DQ3G+oSiH1ptvtK 5b13xU4LDw8uHketZDpHObuR07ROTMQ= Received: from mail-wr1-f71.google.com (mail-wr1-f71.google.com [209.85.221.71]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-623-OHKvPI4NMh6vyrkf0RmdNg-1; Wed, 17 May 2023 13:46:32 -0400 X-MC-Unique: OHKvPI4NMh6vyrkf0RmdNg-1 Received: by mail-wr1-f71.google.com with SMTP id ffacd0b85a97d-3062dedf7d9so752973f8f.2 for ; Wed, 17 May 2023 10:46:32 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1684345591; x=1686937591; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=VUI5gP+VGMEuKYZHL551pN3vhWgbV23BFYT1rHteqTM=; b=X4lEKUBkgicrssPgd+Y9OEGE58m/B7CcESKKko0e0+qY6/kVIeLiZU+pJTXxmxb9B6 Y7qOQ/OYZbIxliIUalLzWdK8Plzif8wbcgLTm9ADnuvhUl5UzkKrYHPGwqCMEc+SLtCv NZmUbeKDL9NJUITUUUVcrLFOoRrYwdtcDFtDG2k0TbZX8Q+SsQ/nBw98yJAj9GV9aPRe 9OFA6ytC6Ey9q61jaVIR+G5yIxU1BpCNfmx8MpQ7SZxi9VdYbBHpwQzKsg6Qypsd6PCq giMlbcxKQltZ/LpX4rcAbrqqdtvAAUE0W7y7ku2HHottscqJZTIvuhRZUimYB7YMENhO paDw== X-Gm-Message-State: AC+VfDwVbCNEEQc3um0IVIKAGs61FXsxGykK+qd+39gIuZKalijKz8ti maat27j4EpSCTWqjuYQxowCWZ0zV7/rudyIWUpPEmL6Yejk5hDoKrZ0jcuLuTORZO3qcOsrB/oQ GX5oua3lidcIy/fX2gR4YozFHD6Gf43gRx6EOVwOWj3pq6SF6rDVR3/mfUfF9sh4vjTx6rHkciN U= X-Received: by 2002:a5d:6e01:0:b0:2f8:3225:2bc2 with SMTP id h1-20020a5d6e01000000b002f832252bc2mr1145756wrz.41.1684345590882; Wed, 17 May 2023 10:46:30 -0700 (PDT) X-Google-Smtp-Source: ACHHUZ6aAKGWc7z9keuDsu7vgic67FbU9a9ol295noQS6AOur4l/MFnec0T+fOxTAsrzTpkm8fWzWQ== X-Received: by 2002:a5d:6e01:0:b0:2f8:3225:2bc2 with SMTP id h1-20020a5d6e01000000b002f832252bc2mr1145740wrz.41.1684345590456; Wed, 17 May 2023 10:46:30 -0700 (PDT) Received: from [192.168.10.118] ([2001:b07:6468:f312:9af8:e5f5:7516:fa89]) by smtp.gmail.com with ESMTPSA id s13-20020adff80d000000b00307bc4e39e5sm3473093wrp.117.2023.05.17.10.46.29 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 17 May 2023 10:46:29 -0700 (PDT) From: Paolo Bonzini To: qemu-devel@nongnu.org Cc: John Snow , =?utf-8?q?Daniel_P_=2E_Berrang=C3=A9?= Subject: [PULL 41/68] Python: Drop support for Python 3.6 Date: Wed, 17 May 2023 19:44:53 +0200 Message-Id: <20230517174520.887405-42-pbonzini@redhat.com> X-Mailer: git-send-email 2.40.1 In-Reply-To: <20230517174520.887405-1-pbonzini@redhat.com> References: <20230517174520.887405-1-pbonzini@redhat.com> MIME-Version: 1.0 Received-SPF: pass client-ip=170.10.129.124; envelope-from=pbonzini@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Python 3.6 was EOL 2021-12-31. Newer versions of upstream libraries have begun dropping support for this version and it is becoming more cumbersome to support. Avocado-framework and qemu.qmp each have their own reasons for wanting to drop Python 3.6, but won't until QEMU does. Versions of Python available in our supported build platforms as of today, with optional versions available in parentheses: openSUSE Leap 15.4: 3.6.15 (3.9.10, 3.10.2) CentOS Stream 8: 3.6.8 (3.8.13, 3.9.16) CentOS Stream 9: 3.9.13 Fedora 36: 3.10 Fedora 37: 3.11 Debian 11: 3.9.2 Alpine 3.14, 3.15: 3.9.16 Alpine 3.16, 3.17: 3.10.10 Ubuntu 20.04 LTS: 3.8.10 Ubuntu 22.04 LTS: 3.10.4 NetBSD 9.3: 3.9.13* FreeBSD 12.4: 3.9.16 FreeBSD 13.1: 3.9.16 OpenBSD 7.2: 3.9.16 Note: Our VM tests install 3.9 explicitly for FreeBSD and 3.10 for NetBSD; the default for "python" or "python3" in FreeBSD is 3.9.16. NetBSD does not appear to have a default meta-package, but offers several options, the lowest of which is 3.7.15. "python39" appears to be a pre-requisite to one of the other packages we request in tests/vm/netbsd. pip, ensurepip and other Python essentials are currently only available for Python 3.10 for NetBSD. CentOS and OpenSUSE support parallel installation of multiple Python interpreters, and binaries in /usr/bin will always use Python 3.6. However, the newly introduced support for virtual environments ensures that all build steps that execute QEMU Python code use a single interpreter. Since it is safe to under our supported platform policy, bump our minimum supported version of Python to 3.7. Signed-off-by: John Snow Reviewed-by: Daniel P. BerrangĂ© Signed-off-by: Paolo Bonzini Message-Id: <20230511035435.734312-24-jsnow@redhat.com> Signed-off-by: Paolo Bonzini --- configure | 10 +++++----- docs/about/build-platforms.rst | 2 +- python/Makefile | 10 +++++----- python/setup.cfg | 7 +++---- python/tests/minreqs.txt | 2 +- scripts/qapi/mypy.ini | 2 +- 6 files changed, 16 insertions(+), 17 deletions(-) diff --git a/configure b/configure index b92da9e6605b..22e31a46d115 100755 --- a/configure +++ b/configure @@ -617,9 +617,9 @@ esac check_py_version() { - # We require python >= 3.6. + # We require python >= 3.7. # NB: a True python conditional creates a non-zero return code (Failure) - "$1" -c 'import sys; sys.exit(sys.version_info < (3,6))' + "$1" -c 'import sys; sys.exit(sys.version_info < (3,7))' } python= @@ -628,7 +628,7 @@ first_python= if test -z "${PYTHON}"; then # A bare 'python' is traditionally python 2.x, but some distros # have it as python 3.x, so check in both places. - for binary in python3 python python3.11 python3.10 python3.9 python3.8 python3.7 python3.6; do + for binary in python3 python python3.11 python3.10 python3.9 python3.8 python3.7; do if has "$binary"; then python=$(command -v "$binary") if check_py_version "$python"; then @@ -1077,7 +1077,7 @@ then # If first_python is set, there was a binary somewhere even though # it was not suitable. Use it for the error message. if test -n "$first_python"; then - error_exit "Cannot use '$first_python', Python >= 3.6 is required." \ + error_exit "Cannot use '$first_python', Python >= 3.7 is required." \ "Use --python=/path/to/python to specify a supported Python." else error_exit "Python not found. Use --python=/path/to/python" @@ -1090,7 +1090,7 @@ then fi if ! check_py_version "$python"; then - error_exit "Cannot use '$python', Python >= 3.6 is required." \ + error_exit "Cannot use '$python', Python >= 3.7 is required." \ "Use --python=/path/to/python to specify a supported Python." fi diff --git a/docs/about/build-platforms.rst b/docs/about/build-platforms.rst index 89cae5a6bb98..0e2cb9e770d5 100644 --- a/docs/about/build-platforms.rst +++ b/docs/about/build-platforms.rst @@ -98,7 +98,7 @@ Python runtime option of the ``configure`` script to point QEMU to a supported version of the Python runtime. - As of QEMU |version|, the minimum supported version of Python is 3.6. + As of QEMU |version|, the minimum supported version of Python is 3.7. Python build dependencies Some of QEMU's build dependencies are written in Python. Usually these diff --git a/python/Makefile b/python/Makefile index 47560657d27e..7c70dcc8d144 100644 --- a/python/Makefile +++ b/python/Makefile @@ -9,14 +9,14 @@ help: @echo "make check-minreqs:" @echo " Run tests in the minreqs virtual environment." @echo " These tests use the oldest dependencies." - @echo " Requires: Python 3.6" - @echo " Hint (Fedora): 'sudo dnf install python3.6'" + @echo " Requires: Python 3.7" + @echo " Hint (Fedora): 'sudo dnf install python3.7'" @echo "" @echo "make check-tox:" @echo " Run tests against multiple python versions." @echo " These tests use the newest dependencies." - @echo " Requires: Python 3.6 - 3.10, and tox." - @echo " Hint (Fedora): 'sudo dnf install python3-tox python3.10'" + @echo " Requires: Python 3.7 - 3.11, and tox." + @echo " Hint (Fedora): 'sudo dnf install python3-tox python3.11'" @echo " The variable QEMU_TOX_EXTRA_ARGS can be use to pass extra" @echo " arguments to tox". @echo "" @@ -59,7 +59,7 @@ PIP_INSTALL = pip install --disable-pip-version-check min-venv: $(QEMU_MINVENV_DIR) $(QEMU_MINVENV_DIR)/bin/activate $(QEMU_MINVENV_DIR) $(QEMU_MINVENV_DIR)/bin/activate: setup.cfg tests/minreqs.txt @echo "VENV $(QEMU_MINVENV_DIR)" - @python3.6 -m venv $(QEMU_MINVENV_DIR) + @python3.7 -m venv $(QEMU_MINVENV_DIR) @( \ echo "ACTIVATE $(QEMU_MINVENV_DIR)"; \ . $(QEMU_MINVENV_DIR)/bin/activate; \ diff --git a/python/setup.cfg b/python/setup.cfg index fc3fae5b1076..55c0993e70b3 100644 --- a/python/setup.cfg +++ b/python/setup.cfg @@ -14,7 +14,6 @@ classifiers = Natural Language :: English Operating System :: OS Independent Programming Language :: Python :: 3 :: Only - Programming Language :: Python :: 3.6 Programming Language :: Python :: 3.7 Programming Language :: Python :: 3.8 Programming Language :: Python :: 3.9 @@ -23,7 +22,7 @@ classifiers = Typing :: Typed [options] -python_requires = >= 3.6 +python_requires = >= 3.7 packages = qemu.qmp qemu.machine @@ -77,7 +76,7 @@ exclude = __pycache__, [mypy] strict = True -python_version = 3.6 +python_version = 3.7 warn_unused_configs = True namespace_packages = True warn_unused_ignores = False @@ -199,7 +198,7 @@ multi_line_output=3 # of python available on your system to run this test. [tox:tox] -envlist = py36, py37, py38, py39, py310, py311 +envlist = py37, py38, py39, py310, py311 skip_missing_interpreters = true [testenv] diff --git a/python/tests/minreqs.txt b/python/tests/minreqs.txt index 7ecf5e7fe483..10b181d43a6b 100644 --- a/python/tests/minreqs.txt +++ b/python/tests/minreqs.txt @@ -1,5 +1,5 @@ # This file lists the ***oldest possible dependencies*** needed to run -# "make check" successfully under ***Python 3.6***. It is used primarily +# "make check" successfully under ***Python 3.7***. It is used primarily # by GitLab CI to ensure that our stated minimum versions in setup.cfg # are truthful and regularly validated. # diff --git a/scripts/qapi/mypy.ini b/scripts/qapi/mypy.ini index 662535642974..3463307ddc72 100644 --- a/scripts/qapi/mypy.ini +++ b/scripts/qapi/mypy.ini @@ -1,7 +1,7 @@ [mypy] strict = True disallow_untyped_calls = False -python_version = 3.6 +python_version = 3.7 [mypy-qapi.schema] disallow_untyped_defs = False From patchwork Wed May 17 17:44:54 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Paolo Bonzini X-Patchwork-Id: 1782837 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=nongnu.org (client-ip=209.51.188.17; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=) Authentication-Results: legolas.ozlabs.org; dkim=pass (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=GevTjJH9; dkim-atps=neutral Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-ECDSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4QM0vZ0XWrz20dn for ; Thu, 18 May 2023 03:48:26 +1000 (AEST) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1pzLF9-0002Wy-2w; Wed, 17 May 2023 13:47:22 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pzLEV-00007G-LO for qemu-devel@nongnu.org; Wed, 17 May 2023 13:46:40 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pzLES-0004ME-QP for qemu-devel@nongnu.org; Wed, 17 May 2023 13:46:38 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1684345595; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=550I0xQYvIiLjv/UoYMa94GFX66P9Jea4TA9rxktSMU=; b=GevTjJH9F2I4SyV3Hz3FeW1AmMJ4x4zeGfFXHJLTBXkaJdB4nQEHvyczqs0eENipV60pU/ cHvJS/p3e5BpGnmE4mPHJZ5KNhdXQ9DIwfe9zOqZ9cx33MAHoX9K92sflzJKUPzaFc5Brz Rp8kSTLf/fIMXa6LHoRhi32pn0dCIwQ= Received: from mail-wr1-f69.google.com (mail-wr1-f69.google.com [209.85.221.69]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-322-5QXc-1RYNF6JZ9IEVVSkBg-1; Wed, 17 May 2023 13:46:33 -0400 X-MC-Unique: 5QXc-1RYNF6JZ9IEVVSkBg-1 Received: by mail-wr1-f69.google.com with SMTP id ffacd0b85a97d-30940b01998so428993f8f.3 for ; Wed, 17 May 2023 10:46:33 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1684345592; x=1686937592; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=550I0xQYvIiLjv/UoYMa94GFX66P9Jea4TA9rxktSMU=; b=P3ggUi//21IrUWHLoZ3XpFtFxvaZj8NdaYefilQY88hkE6eMGINEx/C+F3NMppEJMD aPHyuM56jbL9rWlKDN1xkQrMKCBzftmoIRPtcvd/UTu5NFqK9TlP4roT4xIOgGF1mhdk UUZ1PhaAUM7cHdOoIifNCKUV9IBMoXhkdXezhC/CCAiOxxV4b8tHkVeuei6kypgVzRCC u/J2NzaIXT1cYIEVZWF1kgGCnfhP56YjjbMT3jfyajAp9vmueZo1XlPAvlp1MuHmQ5vo IyGAbzAobxmgiqu61muChZJ+LeWjHOY2/6wS4jfrUS6i0f3x5AslNkrVrD/Hg60aci4d B0cA== X-Gm-Message-State: AC+VfDwU0lZAIJhshNHjeBoNvwQFENswN/uc2xFqj1Kr3T3xjQcKIKqg 4h6HB6DRPLVxIGAhSjXE8d+sK7+XEd6Q41yqQlso3zUo11Og9ERu4EoyGxyPlUXz89x5PJWUREB umHoel7YMYbIISbPHor68CnkPROsiTUDhDSjENz9rD9HUI3AnGHwjnYiAO+FTlmr2l9G+xuiOHi s= X-Received: by 2002:adf:ef03:0:b0:306:b3f9:e2c9 with SMTP id e3-20020adfef03000000b00306b3f9e2c9mr1323190wro.6.1684345592496; Wed, 17 May 2023 10:46:32 -0700 (PDT) X-Google-Smtp-Source: ACHHUZ7tjSzEia+SIEVBbOdjz5Wc4qAuxvQNa2FJo+vbdGpiViIF/0MP1XWF/X6OJqJyWucBiumqPg== X-Received: by 2002:adf:ef03:0:b0:306:b3f9:e2c9 with SMTP id e3-20020adfef03000000b00306b3f9e2c9mr1323175wro.6.1684345592181; Wed, 17 May 2023 10:46:32 -0700 (PDT) Received: from [192.168.10.118] ([2001:b07:6468:f312:9af8:e5f5:7516:fa89]) by smtp.gmail.com with ESMTPSA id d3-20020adfe883000000b00306299be5a2sm3601283wrm.72.2023.05.17.10.46.30 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 17 May 2023 10:46:31 -0700 (PDT) From: Paolo Bonzini To: qemu-devel@nongnu.org Cc: John Snow , =?utf-8?q?Philippe_Mathieu-Daud=C3=A9?= Subject: [PULL 42/68] configure: Add courtesy hint to Python version failure message Date: Wed, 17 May 2023 19:44:54 +0200 Message-Id: <20230517174520.887405-43-pbonzini@redhat.com> X-Mailer: git-send-email 2.40.1 In-Reply-To: <20230517174520.887405-1-pbonzini@redhat.com> References: <20230517174520.887405-1-pbonzini@redhat.com> MIME-Version: 1.0 Received-SPF: pass client-ip=170.10.133.124; envelope-from=pbonzini@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org From: John Snow If we begin requiring Python 3.7+, a few platforms are going to need to install an additional Python interpreter package. As a courtesy to the user, suggest the optional package they might need to install. This will hopefully minimize any downtime caused by the change in Python dependency. Signed-off-by: John Snow Reviewed-by: Philippe Mathieu-DaudĂ© Message-Id: <20230221012456.2607692-3-jsnow@redhat.com> Signed-off-by: Paolo Bonzini Message-Id: <20230511035435.734312-25-jsnow@redhat.com> Signed-off-by: Paolo Bonzini --- configure | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-) diff --git a/configure b/configure index 22e31a46d115..52eca1cd4c53 100755 --- a/configure +++ b/configure @@ -1091,7 +1091,10 @@ fi if ! check_py_version "$python"; then error_exit "Cannot use '$python', Python >= 3.7 is required." \ - "Use --python=/path/to/python to specify a supported Python." + "Use --python=/path/to/python to specify a supported Python." \ + "Maybe try:" \ + " openSUSE Leap 15.3+: zypper install python39" \ + " CentOS 8: dnf install python38" fi # Resolve PATH From patchwork Wed May 17 17:44:55 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Paolo Bonzini X-Patchwork-Id: 1782867 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=nongnu.org (client-ip=209.51.188.17; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=) Authentication-Results: legolas.ozlabs.org; dkim=pass (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=SK1qlQCb; dkim-atps=neutral Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-ECDSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4QM16M2FPFz20dX for ; Thu, 18 May 2023 03:57:47 +1000 (AEST) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1pzLEZ-0000PA-9K; Wed, 17 May 2023 13:46:43 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pzLEW-00009O-VK for qemu-devel@nongnu.org; Wed, 17 May 2023 13:46:41 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.129.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pzLEV-0004Mo-C9 for qemu-devel@nongnu.org; Wed, 17 May 2023 13:46:40 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1684345597; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=PP9Xn2n4ChTkZoiiBSEFm8Q1rGJycAE9rbs4ej+xjbs=; b=SK1qlQCbB9h71btmdb8x1fxE4k3S24UPYBGL5aKXWAALnQPVm2vQfeRha+FJ7Y4Q1c2i9w fRF7Z18yHmL5vWiUTRhqj8QWCFCKR4F/yDwSJOY9dGj/vTNFAG8TKQ/oQgdQmK2eceZDtz G8VEhIhaO7DZ8w481InQHp0U0Otw2bc= Received: from mail-wm1-f71.google.com (mail-wm1-f71.google.com [209.85.128.71]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-641-80YKyLFAO2uSE2QvQFyjSg-1; Wed, 17 May 2023 13:46:35 -0400 X-MC-Unique: 80YKyLFAO2uSE2QvQFyjSg-1 Received: by mail-wm1-f71.google.com with SMTP id 5b1f17b1804b1-3f4221cd284so4439395e9.0 for ; Wed, 17 May 2023 10:46:35 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1684345594; x=1686937594; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=PP9Xn2n4ChTkZoiiBSEFm8Q1rGJycAE9rbs4ej+xjbs=; b=CGYSCTKV8vVkkdCGVaMndH/tTECrhxgcaeosb0dM64hYseHCv+3LOdAIjPaG/cr97C hnHnsa7qDVouhW+vZjUMxYiJHZnh2kX9fdxbHKEHZ/qzuaYZKGECNdHNajfH34brK2Eo sb9AzbMJTB4lw5CLayCcnJO7YtJoyxVcMnN8izyAt2beSMCNT4IhWUM26FIa0Lx4ubBf R8x8DPzfC3okoFyKov3VWnM9krKkdDNe4uaHPJWIzIj4Zokw54486pdQ4HupH/9EOPnV XblDIJ1FzhqbOD/PFl7KDfadGK9m9fs9QTZBIFfqxYan3cV8Mv2BMLq2ajdmELKovacU 0BSA== X-Gm-Message-State: AC+VfDym7rgn9zZwxfJD3IciFbni9g0lVL/1/bBCGfFLa5G/giaaSJOC mJgoA7fwRyRzlc+77MEIZ6hYrE+gDeRLlelM1gRKFn+nZgimdfaVYG1JJ5tuGG8y5T6cI+N+K+J rGop5zp4TH0o9j7qg1Rmi4a3rt2jFFSLCQo1rh0BwxwzhdS+gY59Za3FmMqSKNBaZqnL1TwKdd2 8= X-Received: by 2002:a7b:cd04:0:b0:3f5:d0ba:3c1a with SMTP id f4-20020a7bcd04000000b003f5d0ba3c1amr1817495wmj.4.1684345593986; Wed, 17 May 2023 10:46:33 -0700 (PDT) X-Google-Smtp-Source: ACHHUZ6fJcUAbKBIvSP5I/Gw0xVF8wuxHuhJyBkokoMOm+JImIsgL0DLZiBIeikRUUrAu4q36GCb7w== X-Received: by 2002:a7b:cd04:0:b0:3f5:d0ba:3c1a with SMTP id f4-20020a7bcd04000000b003f5d0ba3c1amr1817489wmj.4.1684345593755; Wed, 17 May 2023 10:46:33 -0700 (PDT) Received: from [192.168.10.118] ([2001:b07:6468:f312:9af8:e5f5:7516:fa89]) by smtp.gmail.com with ESMTPSA id m9-20020a7bca49000000b003f3157988f8sm2903585wml.26.2023.05.17.10.46.32 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 17 May 2023 10:46:33 -0700 (PDT) From: Paolo Bonzini To: qemu-devel@nongnu.org Cc: John Snow Subject: [PULL 43/68] mkvenv: mark command as required Date: Wed, 17 May 2023 19:44:55 +0200 Message-Id: <20230517174520.887405-44-pbonzini@redhat.com> X-Mailer: git-send-email 2.40.1 In-Reply-To: <20230517174520.887405-1-pbonzini@redhat.com> References: <20230517174520.887405-1-pbonzini@redhat.com> MIME-Version: 1.0 Received-SPF: pass client-ip=170.10.129.124; envelope-from=pbonzini@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org This is only available in Python 3.7+. Signed-off-by: Paolo Bonzini Signed-off-by: John Snow Message-Id: <20230511035435.734312-26-jsnow@redhat.com> Signed-off-by: Paolo Bonzini --- python/scripts/mkvenv.py | 1 + 1 file changed, 1 insertion(+) diff --git a/python/scripts/mkvenv.py b/python/scripts/mkvenv.py index 558619910efd..8c036c019aaf 100644 --- a/python/scripts/mkvenv.py +++ b/python/scripts/mkvenv.py @@ -854,6 +854,7 @@ def main() -> int: subparsers = parser.add_subparsers( title="Commands", dest="command", + required=True, metavar="command", help="Description", ) From patchwork Wed May 17 17:44:56 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Paolo Bonzini X-Patchwork-Id: 1782869 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=nongnu.org (client-ip=209.51.188.17; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=) Authentication-Results: legolas.ozlabs.org; dkim=pass (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=L7+aquQo; dkim-atps=neutral Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-ECDSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4QM16T4LgCz20dq for ; Thu, 18 May 2023 03:57:53 +1000 (AEST) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1pzLF2-00029X-P4; Wed, 17 May 2023 13:47:12 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pzLEY-0000Jn-A6 for qemu-devel@nongnu.org; Wed, 17 May 2023 13:46:42 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pzLEW-0004NO-Nj for qemu-devel@nongnu.org; Wed, 17 May 2023 13:46:42 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1684345600; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=zJN2jANxJFA4WG6/RjjsAlRU9ohw2knjG165asZhq/Q=; b=L7+aquQoJyFaJThQe7m215qtu5keHy/Wl5G8XcvMHh8beqDOZa01oOoNGPRBHwDCAB2k5f WHw7zLFKud8ZRLTIlbbnRwmGk9JaECQk82XnARCpY2S13I3OlBb6+6rKfLCIP0jYmnR8Gp 8BZGmNpRA1/9ywFe13JWytMkh4HSdRo= Received: from mail-wr1-f69.google.com (mail-wr1-f69.google.com [209.85.221.69]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-8-t4SnR1jiPFaoFv6F82LmuQ-1; Wed, 17 May 2023 13:46:37 -0400 X-MC-Unique: t4SnR1jiPFaoFv6F82LmuQ-1 Received: by mail-wr1-f69.google.com with SMTP id ffacd0b85a97d-30479b764f9so451450f8f.0 for ; Wed, 17 May 2023 10:46:37 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1684345595; x=1686937595; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=zJN2jANxJFA4WG6/RjjsAlRU9ohw2knjG165asZhq/Q=; b=MyVACe7heE5Wq1ZhULv6Pr2jlBjF03B0m55es8h1FN7GXJjGdUYl9x96KwZNrq7npc bhZ5fRO6abKVRvZIcZPaQS2GmtBnl4lgLmVg+ohvXyu/2mjQ/cSlo1ghd5zrHFnT/XQK rFJ39joW+0aUtxhRaWubPSvoG1Nn004NpldQT0A4M+lHfrjZ3dUHFmCjfsKrV7H7v+s7 U29v5Ne8aMhLPQPw7wIPVw853w1KBQogRRrczaB2M2VI5dnlc/DDD/vIO3eg0vbD3p9+ k/JIXVZIIOTsPmAmudFPiBv9FVQIRPGDqnGmoh7ArkjnWk4d/7JO3Hs8Y7SlO0XRWi0q OVTQ== X-Gm-Message-State: AC+VfDwQaW2LVj8zApmE7oVvQ9936UtfK1xc8OhSt08m6ydyd2o7WaLV Cg7Hcw/RDRfvdmR/pDWv76n+ulOz2PC4h78fsjmV2jj9i/j+97sMoENkbZL+IEdmLpxLwmRMXlt +eIOSpwac5pcuzCPoRsP50Z20DM71oSedFdzmweyFysU4BNNb3gm11RXXEj+0KfMs60qh2z9qxQ g= X-Received: by 2002:a05:6000:12cc:b0:2fe:6b1e:3818 with SMTP id l12-20020a05600012cc00b002fe6b1e3818mr1274583wrx.51.1684345595600; Wed, 17 May 2023 10:46:35 -0700 (PDT) X-Google-Smtp-Source: ACHHUZ739Ri/uf7NjcD04MzY4r/F7HLDiA40TYeqtd4MxUm+eWe5jUJPiGemAZhAf/8OHf/4x3s3cQ== X-Received: by 2002:a05:6000:12cc:b0:2fe:6b1e:3818 with SMTP id l12-20020a05600012cc00b002fe6b1e3818mr1274573wrx.51.1684345595243; Wed, 17 May 2023 10:46:35 -0700 (PDT) Received: from [192.168.10.118] ([2001:b07:6468:f312:9af8:e5f5:7516:fa89]) by smtp.gmail.com with ESMTPSA id e2-20020a5d4e82000000b00309257ad16csm3383089wru.29.2023.05.17.10.46.34 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 17 May 2023 10:46:34 -0700 (PDT) From: Paolo Bonzini To: qemu-devel@nongnu.org Cc: John Snow Subject: [PULL 44/68] python: bump some of the dependencies Date: Wed, 17 May 2023 19:44:56 +0200 Message-Id: <20230517174520.887405-45-pbonzini@redhat.com> X-Mailer: git-send-email 2.40.1 In-Reply-To: <20230517174520.887405-1-pbonzini@redhat.com> References: <20230517174520.887405-1-pbonzini@redhat.com> MIME-Version: 1.0 Received-SPF: pass client-ip=170.10.133.124; envelope-from=pbonzini@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org The version of pyflakes that is listed in python/tests/minreqs.txt breaks on Python 3.8 with the following message: AttributeError: 'FlakesChecker' object has no attribute 'CONSTANT' Now that we do not support EOL'd Python versions anymore, we can update to newer, fixed versions. It is a good time to do so, before Python packages start dropping support for Python 3.7 as well! The new mypy is also a bit smarter about which packages are actually being used, so remove the now-unnecessary sections from setup.cfg. Signed-off-by: Paolo Bonzini Signed-off-by: John Snow Message-Id: <20230511035435.734312-27-jsnow@redhat.com> Signed-off-by: Paolo Bonzini --- python/setup.cfg | 10 ++-------- python/tests/minreqs.txt | 14 +++++++------- 2 files changed, 9 insertions(+), 15 deletions(-) diff --git a/python/setup.cfg b/python/setup.cfg index 55c0993e70b3..5abb7d30ad42 100644 --- a/python/setup.cfg +++ b/python/setup.cfg @@ -36,11 +36,11 @@ packages = devel = avocado-framework >= 90.0 distlib >= 0.3.6 - flake8 >= 3.6.0 + flake8 >= 5.0.4 fusepy >= 2.0.4 isort >= 5.1.2 mypy >= 0.780 - pylint >= 2.8.0 + pylint >= 2.17.3 tox >= 3.18.0 urwid >= 2.1.2 urwid-readline >= 0.13 @@ -124,12 +124,6 @@ ignore_missing_imports = True [mypy-distlib.version] ignore_missing_imports = True -[mypy-pip] -ignore_missing_imports = True - -[mypy-pip._vendor] -ignore_missing_imports = True - [mypy-pip._vendor.distlib] ignore_missing_imports = True diff --git a/python/tests/minreqs.txt b/python/tests/minreqs.txt index 10b181d43a6b..1ce72cef6d80 100644 --- a/python/tests/minreqs.txt +++ b/python/tests/minreqs.txt @@ -26,23 +26,23 @@ fusepy==2.0.4 avocado-framework==90.0 # Linters -flake8==3.6.0 +flake8==5.0.4 isort==5.1.2 mypy==0.780 -pylint==2.8.0 +pylint==2.17.3 # Transitive flake8 dependencies -mccabe==0.6.0 -pycodestyle==2.4.0 -pyflakes==2.0.0 +mccabe==0.7.0 +pycodestyle==2.9.1 +pyflakes==2.5.0 # Transitive mypy dependencies mypy-extensions==0.4.3 typed-ast==1.4.0 -typing-extensions==3.7.4 +typing-extensions==4.5.0 # Transitive pylint dependencies -astroid==2.5.4 +astroid==2.15.4 lazy-object-proxy==1.4.0 toml==0.10.0 wrapt==1.12.1 From patchwork Wed May 17 17:44:57 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Paolo Bonzini X-Patchwork-Id: 1782856 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=nongnu.org (client-ip=209.51.188.17; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=) Authentication-Results: legolas.ozlabs.org; dkim=pass (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=QSYeWDhD; dkim-atps=neutral Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-ECDSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4QM12d5XBHz20KF for ; Thu, 18 May 2023 03:54:33 +1000 (AEST) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1pzLFm-0003C6-QQ; Wed, 17 May 2023 13:47:58 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pzLEY-0000JA-8D for qemu-devel@nongnu.org; Wed, 17 May 2023 13:46:42 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pzLEW-0004NL-NS for qemu-devel@nongnu.org; Wed, 17 May 2023 13:46:41 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1684345600; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=wZLbxMCZ5/pZsSEQbytFR8Ifx73pTURrqmwPuyeAL9Q=; b=QSYeWDhDJtJSQXTWvAcmRf2kcNmVX+xWXyliCsw6YF+K5KonkKAf4ZxPeNFuDV0sufn6r4 rO4lXrbG8Hd8tIC4yEuOXKJBKm62mg+K9h9b459jpd3X0hi6lPa3jp0uBMYdzr/4ch8Zsv Vog+WWNnPjAHZT0wm1hMXQy7fWEqMaI= Received: from mail-wm1-f71.google.com (mail-wm1-f71.google.com [209.85.128.71]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-518-TBK0aT3qOTuEqQyaGkPU8g-1; Wed, 17 May 2023 13:46:38 -0400 X-MC-Unique: TBK0aT3qOTuEqQyaGkPU8g-1 Received: by mail-wm1-f71.google.com with SMTP id 5b1f17b1804b1-3f42b36733aso4440135e9.3 for ; Wed, 17 May 2023 10:46:38 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1684345597; x=1686937597; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=wZLbxMCZ5/pZsSEQbytFR8Ifx73pTURrqmwPuyeAL9Q=; b=Uv7IB5lXvDtO82r/vfBPoUmOIA/Vz5ZWGr70iPE6nVobX/L20p1azAJcEpGGTHI5ss jiszeudv9xr7hUTzl7v1Jw2pq+OjZWvW77zyhtuYU2mnU+xwhnojLuRX3fmaONm1pVH3 dfl/wlgkArNVC4uy0I1qc0fPRzlz3xYaM49XtzvhmzLgOG57p/6jjIfKYVHj5PBXr1VX Tms4/T4uQvcRn5IMxNc/Lb55Z0CRetzMeahO1rMjIS/6U/rG7ObC5oGy8IrIYg5k1tec 4L1RMjsOxhYpoYsvPKWslSjMDoNZWlgHm8jdBXQnGoDVSEb4tXiMeLHSklaWKitlaxus +Q/Q== X-Gm-Message-State: AC+VfDxM3n5F7OdmMX9pLEhUZN4km5vkWubqTagc3kxG3xwSpeGWS1Wk 1vBZcxvRUy2gec1v5hGctn+VviqGVxrk6R2Yt3oZBQK1RwzNM1B8CJoHmC63SkmD4vG3ezlds8a sVBTudFLUW0CJgtzEy/gHpcyXhgydnIbCf6e0PaK/rRsjqofaD7O0B1aK+2fTKx/kYiLdkoEOU5 s= X-Received: by 2002:a05:600c:d2:b0:3f4:28db:f609 with SMTP id u18-20020a05600c00d200b003f428dbf609mr23676636wmm.36.1684345596970; Wed, 17 May 2023 10:46:36 -0700 (PDT) X-Google-Smtp-Source: ACHHUZ5qwsBZFt0MtydiRx4MfRDbu1I5zSDM8gAHa/XfB0selH0sbILOx3G31FgFAm0xz0+gf8boEQ== X-Received: by 2002:a05:600c:d2:b0:3f4:28db:f609 with SMTP id u18-20020a05600c00d200b003f428dbf609mr23676628wmm.36.1684345596646; Wed, 17 May 2023 10:46:36 -0700 (PDT) Received: from [192.168.10.118] ([2001:b07:6468:f312:9af8:e5f5:7516:fa89]) by smtp.gmail.com with ESMTPSA id o4-20020a5d4a84000000b003068f5cca8csm3386711wrq.94.2023.05.17.10.46.35 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 17 May 2023 10:46:35 -0700 (PDT) From: Paolo Bonzini To: qemu-devel@nongnu.org Subject: [PULL 45/68] meson: regenerate meson-buildoptions.sh Date: Wed, 17 May 2023 19:44:57 +0200 Message-Id: <20230517174520.887405-46-pbonzini@redhat.com> X-Mailer: git-send-email 2.40.1 In-Reply-To: <20230517174520.887405-1-pbonzini@redhat.com> References: <20230517174520.887405-1-pbonzini@redhat.com> MIME-Version: 1.0 Received-SPF: pass client-ip=170.10.133.124; envelope-from=pbonzini@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Signed-off-by: Paolo Bonzini --- scripts/meson-buildoptions.sh | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/scripts/meson-buildoptions.sh b/scripts/meson-buildoptions.sh index fedb93ada69e..10c4969a013e 100644 --- a/scripts/meson-buildoptions.sh +++ b/scripts/meson-buildoptions.sh @@ -1,8 +1,8 @@ # This file is generated by meson-buildoptions.py, do not edit! meson_options_help() { - printf "%s\n" ' --audio-drv-list=CHOICES Set audio driver list [default] (choices: al' - printf "%s\n" ' sa/coreaudio/default/dsound/jack/oss/pa/' - printf "%s\n" ' pipewire/sdl/sndio)' + printf "%s\n" ' --audio-drv-list=CHOICES Set audio driver list [default] (choices: alsa/co' + printf "%s\n" ' reaudio/default/dsound/jack/oss/pa/pipewire/sdl/s' + printf "%s\n" ' ndio)' printf "%s\n" ' --block-drv-ro-whitelist=VALUE' printf "%s\n" ' set block driver read-only whitelist (by default' printf "%s\n" ' affects only QEMU, not tools like qemu-img)' From patchwork Wed May 17 17:44:58 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Paolo Bonzini X-Patchwork-Id: 1782889 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=nongnu.org (client-ip=209.51.188.17; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=) Authentication-Results: legolas.ozlabs.org; dkim=pass (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=J5yqCuur; dkim-atps=neutral Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-ECDSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4QM1Fv6dlRz20dn for ; Thu, 18 May 2023 04:04:19 +1000 (AEST) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1pzLGi-0004Bt-SW; Wed, 17 May 2023 13:49:00 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pzLEa-0000cl-3D for qemu-devel@nongnu.org; Wed, 17 May 2023 13:46:44 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pzLEY-0004O9-CP for qemu-devel@nongnu.org; Wed, 17 May 2023 13:46:43 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1684345601; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=FUOZpiyE32GQEdNrdxVkXRqUJswyfTYYATcn6Mk6EHo=; b=J5yqCuuri8X4DnU9nZnxrV4c0yVkXYmJZJdZowtpPOedWpWdPhMOSHJwlimXS4iuTe/XH0 h0pePaNqawWRWZXgbVLSapZQwO5TiwjmamRLsjqRynQVlhz++C18fYb1Nr1/12DuqOeBRI i5yOzNHMTrTYvGeruEc3hp1BLgjtgQU= Received: from mail-wr1-f70.google.com (mail-wr1-f70.google.com [209.85.221.70]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-492-K3XnuCrdN4iW8lE2uhVNgQ-1; Wed, 17 May 2023 13:46:40 -0400 X-MC-Unique: K3XnuCrdN4iW8lE2uhVNgQ-1 Received: by mail-wr1-f70.google.com with SMTP id ffacd0b85a97d-30634323dfeso490456f8f.3 for ; Wed, 17 May 2023 10:46:40 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1684345599; x=1686937599; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=FUOZpiyE32GQEdNrdxVkXRqUJswyfTYYATcn6Mk6EHo=; b=Z+MGchO9q2Q3NUm5jaEKSEOb6Y3dyWLDOiL7CtasQ084VJ+G6z9Zh6XrCu1sfQrlUP u2Y7AGPdANNtsUIoqoLmuYXYe8n7/JLi8MIM1Ru70/NOEeBEdILafaDzAUknIqsFboOi VQbhuhx1wr3EZxLxbqdfZRJA57ZIlT1zynqFJk8R3ffwA8n0AhqobNPb+vobwxTzL/OI aaKxMKxOmv6YoDlcZcDPT5q6vOmnABiQwNUeROTG08Gcxs9GsewItaz6YNPoz9R4kSaS tr1Vzzg/b9bmR2cGOhHlBrtYLwyKdCe6VyeDU5zQNEhBAZB8R+yRr7lyX85Oho4H+Hdx jhnQ== X-Gm-Message-State: AC+VfDyPsysPlWXMk7xZ3vWQB9WZQNf0yFDa96/K/4XMCOhgv53nnAdE IkfEpK2fZPPfx53BcPuWi1dTC6V/7+JRkCm2FiQ1hJGrEEW3Y0Xs02Vt7pN25Ka457ZZ+3niO4c lSzwv4zhgCajA7lmQJci+11dSYOL1KrEVXls9SHFldjk97k6M1wEHxiwO0mEOJY8F4dbn8LeY+M Q= X-Received: by 2002:adf:f2c3:0:b0:309:436a:fc2f with SMTP id d3-20020adff2c3000000b00309436afc2fmr1067185wrp.57.1684345598846; Wed, 17 May 2023 10:46:38 -0700 (PDT) X-Google-Smtp-Source: ACHHUZ5mhRSc0b3myuBOexfJPpWFIh2DwT0MoxCEtPQh0tMfdvmLL896zzY8cH1qUV8pgySRu2o59Q== X-Received: by 2002:adf:f2c3:0:b0:309:436a:fc2f with SMTP id d3-20020adff2c3000000b00309436afc2fmr1067174wrp.57.1684345598398; Wed, 17 May 2023 10:46:38 -0700 (PDT) Received: from [192.168.10.118] ([2001:b07:6468:f312:9af8:e5f5:7516:fa89]) by smtp.gmail.com with ESMTPSA id v5-20020a5d6785000000b002fb60c7995esm3425516wru.8.2023.05.17.10.46.37 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 17 May 2023 10:46:37 -0700 (PDT) From: Paolo Bonzini To: qemu-devel@nongnu.org Subject: [PULL 46/68] meson: require 0.63.0 Date: Wed, 17 May 2023 19:44:58 +0200 Message-Id: <20230517174520.887405-47-pbonzini@redhat.com> X-Mailer: git-send-email 2.40.1 In-Reply-To: <20230517174520.887405-1-pbonzini@redhat.com> References: <20230517174520.887405-1-pbonzini@redhat.com> MIME-Version: 1.0 Received-SPF: pass client-ip=170.10.133.124; envelope-from=pbonzini@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org This version allows cleanups in modinfo collection, but they only work with Ninja 1.9.x and 1.8.x is still supported. It also supports the equivalent of QEMU's --static option to configure. The wheel file is bumped to 0.63.3, the last release in the 0.63 branch. Signed-off-by: Paolo Bonzini --- configure | 2 +- meson.build | 2 +- python/scripts/vendor.py | 4 ++-- python/wheels/meson-0.61.5-py3-none-any.whl | Bin 862509 -> 0 bytes python/wheels/meson-0.63.3-py3-none-any.whl | Bin 0 -> 926526 bytes 5 files changed, 4 insertions(+), 4 deletions(-) delete mode 100644 python/wheels/meson-0.61.5-py3-none-any.whl create mode 100644 python/wheels/meson-0.63.3-py3-none-any.whl diff --git a/configure b/configure index 52eca1cd4c53..dbdcf3886d56 100755 --- a/configure +++ b/configure @@ -1136,7 +1136,7 @@ if ! $mkvenv ensure \ $mkvenv_flags \ --dir "${source_path}/python/wheels" \ --diagnose "meson" \ - "meson>=0.61.5" ; + "meson>=0.63.0" ; then exit 1 fi diff --git a/meson.build b/meson.build index 25a4b9f2c106..e3a9db20ffa7 100644 --- a/meson.build +++ b/meson.build @@ -1,4 +1,4 @@ -project('qemu', ['c'], meson_version: '>=0.61.3', +project('qemu', ['c'], meson_version: '>=0.63.0', default_options: ['warning_level=1', 'c_std=gnu11', 'cpp_std=gnu++11', 'b_colorout=auto', 'b_staticpic=false', 'stdsplit=false', 'optimization=2', 'b_pie=true'], version: files('VERSION')) diff --git a/python/scripts/vendor.py b/python/scripts/vendor.py index 23708430ea68..34486a51f445 100755 --- a/python/scripts/vendor.py +++ b/python/scripts/vendor.py @@ -41,8 +41,8 @@ def main() -> int: parser.parse_args() packages = { - "meson==0.61.5": - "58c2ddb5f885da0e929f15d89f38d8a7d97f981f56815bcba008414f8511f59a", + "meson==0.63.3": + "d677b809c4895dcbaac9bf6c43703fcb3609a4b24c6057c78f828590049cf43a", } vendor_dir = Path(__file__, "..", "..", "wheels").resolve() From patchwork Wed May 17 17:44:59 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Paolo Bonzini X-Patchwork-Id: 1782875 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=nongnu.org (client-ip=209.51.188.17; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=) Authentication-Results: legolas.ozlabs.org; dkim=pass (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=QSblVqah; dkim-atps=neutral Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-ECDSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4QM1861jVqz20dX for ; Thu, 18 May 2023 03:59:18 +1000 (AEST) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1pzLGv-0004vU-0t; Wed, 17 May 2023 13:49:09 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pzLEb-0000tD-Vc for qemu-devel@nongnu.org; Wed, 17 May 2023 13:46:46 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pzLEZ-0004OX-Rp for qemu-devel@nongnu.org; Wed, 17 May 2023 13:46:45 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1684345603; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=6L8pigL04+7noMI87Ra9LpcpynCzBM12C1I2eXwOcmo=; b=QSblVqahjem5sN21SABqb97wx4NHcyowWqt3gPXmUZccwuiZgdNekA3AEXWNfVuKKJrIWQ NaZ4FW2DMJPmohO/I/XDyM3j6c7x5cM+vejcdJEnxZSpehbt996hVE76YwV28rQooLhWvx ITldHwUqc65NZaapQ9lW7+jMQ7Ggy1Y= Received: from mail-wm1-f72.google.com (mail-wm1-f72.google.com [209.85.128.72]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-496-Z8GK2XaCO6mjQ9TlC9BenA-1; Wed, 17 May 2023 13:46:42 -0400 X-MC-Unique: Z8GK2XaCO6mjQ9TlC9BenA-1 Received: by mail-wm1-f72.google.com with SMTP id 5b1f17b1804b1-3f422dc5fafso7065695e9.0 for ; Wed, 17 May 2023 10:46:41 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1684345600; x=1686937600; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=6L8pigL04+7noMI87Ra9LpcpynCzBM12C1I2eXwOcmo=; b=d6iIybN7GeFq2W7wpmzxhYVEGjd16cFan/P6JjTnNClT8WYlR3lEs1w97aal5Lowjl f4xL1TexJmW8iACu9puoqMG46ex3djZK2Rurq/avrqDt3aXGx/R7HF194BbgSgNCg5tZ b78VPXoiHIxWoyq9Kn/XlZ1FGbG3FvuW1NWRRgXVGlqoX5iwbEZA7Q7ESU3WW5snT3dV OW8b4iealuF/ow0eiGlLSPWIUhSngpFBu1v96ccjkNtvhhZC/7LUnN4kIDJLW7ZrsiXa eqaNAHCOhBngEOpZe/Dvapj6YmThmVatHFH+aWA0b2cY8uJs7aSKRN6HKl4sVXh2UTTE A8ig== X-Gm-Message-State: AC+VfDzq95/7cf31cTrbkZxA32jNmLgXrZLwJsRW4kT9XKzJJ6BupYvq 9YqZQBJkr3aPQW7MRygT/4URIibdO8r40OI93s/Usk0TtAOugjmF82lPPGpN4AriU0VFiT/FCLX DAqlLj0MB+7Qah3RxgLdagBGUXxG5kNy3pNJiK2giPdpxmeZ00G+hbQ/IVJ3OvulW/lPE2aPo68 o= X-Received: by 2002:a1c:4c08:0:b0:3f4:2737:a016 with SMTP id z8-20020a1c4c08000000b003f42737a016mr21134062wmf.29.1684345600201; Wed, 17 May 2023 10:46:40 -0700 (PDT) X-Google-Smtp-Source: ACHHUZ4tehf9GGUAcNkG65GckUySCThEB2d+jhgQR6rApYzgFsGJVZdi8JlvJfNGgbVDFlkq5r3JAA== X-Received: by 2002:a1c:4c08:0:b0:3f4:2737:a016 with SMTP id z8-20020a1c4c08000000b003f42737a016mr21134050wmf.29.1684345599836; Wed, 17 May 2023 10:46:39 -0700 (PDT) Received: from [192.168.10.118] ([2001:b07:6468:f312:9af8:e5f5:7516:fa89]) by smtp.gmail.com with ESMTPSA id z18-20020a1c4c12000000b003f4f8cc4285sm2854423wmf.17.2023.05.17.10.46.38 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 17 May 2023 10:46:39 -0700 (PDT) From: Paolo Bonzini To: qemu-devel@nongnu.org Cc: =?utf-8?q?Marc-Andr=C3=A9_Lureau?= Subject: [PULL 47/68] meson: use prefer_static option Date: Wed, 17 May 2023 19:44:59 +0200 Message-Id: <20230517174520.887405-48-pbonzini@redhat.com> X-Mailer: git-send-email 2.40.1 In-Reply-To: <20230517174520.887405-1-pbonzini@redhat.com> References: <20230517174520.887405-1-pbonzini@redhat.com> MIME-Version: 1.0 Received-SPF: pass client-ip=170.10.133.124; envelope-from=pbonzini@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org The option is new in Meson 0.63 and removes the need to pass "static: true" to all dependency and find_library invocation. Actually cleaning up the invocations is left for a separate patch. Reviewed-by: Marc-AndrĂ© Lureau Signed-off-by: Paolo Bonzini --- configure | 4 +--- docs/devel/build-system.rst | 3 +-- meson.build | 13 +++++-------- qga/meson.build | 2 +- 4 files changed, 8 insertions(+), 14 deletions(-) diff --git a/configure b/configure index dbdcf3886d56..651ada7b6bc3 100755 --- a/configure +++ b/configure @@ -2409,9 +2409,6 @@ fi if test "$solaris" = "yes" ; then echo "CONFIG_SOLARIS=y" >> $config_host_mak fi -if test "$static" = "yes" ; then - echo "CONFIG_STATIC=y" >> $config_host_mak -fi echo "SRC_PATH=$source_path" >> $config_host_mak echo "TARGET_DIRS=$target_list" >> $config_host_mak if test "$modules" = "yes"; then @@ -2654,6 +2651,7 @@ if test "$skip_meson" = no; then # Built-in options test "$bindir" != "bin" && meson_option_add "-Dbindir=$bindir" test "$default_feature" = no && meson_option_add -Dauto_features=disabled + test "$static" = yes && meson_option_add -Dprefer_static=true test "$pie" = no && meson_option_add -Db_pie=false test "$werror" = yes && meson_option_add -Dwerror=true diff --git a/docs/devel/build-system.rst b/docs/devel/build-system.rst index 189472174340..9db18aff159e 100644 --- a/docs/devel/build-system.rst +++ b/docs/devel/build-system.rst @@ -311,8 +311,7 @@ dependency will be used:: sdl_image = not_found if not get_option('sdl_image').auto() or have_system sdl_image = dependency('SDL2_image', required: get_option('sdl_image'), - method: 'pkg-config', - static: enable_static) + method: 'pkg-config') endif This avoids warnings on static builds of user-mode emulators, for example. diff --git a/meson.build b/meson.build index e3a9db20ffa7..ac5bb903d2c6 100644 --- a/meson.build +++ b/meson.build @@ -18,10 +18,7 @@ targetos = host_machine.system() sh = find_program('sh') config_host = keyval.load(meson.current_build_dir() / 'config-host.mak') enable_modules = 'CONFIG_MODULES' in config_host -enable_static = 'CONFIG_STATIC' in config_host - -# Allow both shared and static libraries unless --enable-static -static_kwargs = enable_static ? {'static': true} : {} +static_kwargs = {} cc = meson.get_compiler('c') all_languages = ['c'] @@ -193,7 +190,7 @@ qemu_cflags = config_host['QEMU_CFLAGS'].split() qemu_objcflags = config_host['QEMU_OBJCFLAGS'].split() qemu_ldflags = config_host['QEMU_LDFLAGS'].split() -if enable_static +if get_option('prefer_static') qemu_ldflags += get_option('b_pie') ? '-static-pie' : '-static' endif @@ -841,7 +838,7 @@ if targetos == 'linux' and have_tools and get_option('mpath').allowed() kwargs: static_kwargs) if libmpathpersist.found() mpathlibs += libmpathpersist - if enable_static + if get_option('prefer_static') mpathlibs += cc.find_library('devmapper', required: get_option('mpath'), kwargs: static_kwargs) @@ -1225,7 +1222,7 @@ if not gnutls_crypto.found() # Debian has removed -lgpg-error from libgcrypt-config # as it "spreads unnecessary dependencies" which in # turn breaks static builds... - if gcrypt.found() and enable_static + if gcrypt.found() and get_option('prefer_static') gcrypt = declare_dependency(dependencies: [ gcrypt, cc.find_library('gpg-error', required: true, kwargs: static_kwargs)]) @@ -1657,7 +1654,7 @@ endif # libdw libdw = not_found if not get_option('libdw').auto() or \ - (not enable_static and (have_system or have_user)) + (not get_option('prefer_static') and (have_system or have_user)) libdw = dependency('libdw', method: 'pkg-config', kwargs: static_kwargs, diff --git a/qga/meson.build b/qga/meson.build index ad17dc7dca13..622b5f94a232 100644 --- a/qga/meson.build +++ b/qga/meson.build @@ -22,7 +22,7 @@ have_qga_vss = get_option('qga_vss') \ Then run configure with: --extra-cxxflags="-isystem /path/to/vss/inc/win2003"''') \ .require(midl.found() or widl.found(), error_message: 'VSS support requires midl or widl') \ - .require(not enable_static, + .require(not get_option('prefer_static'), error_message: 'VSS support requires dynamic linking with GLib') \ .allowed() From patchwork Wed May 17 17:45:00 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Paolo Bonzini X-Patchwork-Id: 1782868 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=nongnu.org (client-ip=209.51.188.17; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=) Authentication-Results: legolas.ozlabs.org; dkim=pass (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=T5+Mx4hm; dkim-atps=neutral Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-ECDSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4QM16S59Wtz20dX for ; Thu, 18 May 2023 03:57:52 +1000 (AEST) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1pzLH7-0005TA-O6; Wed, 17 May 2023 13:49:21 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pzLEg-0001cc-4B for qemu-devel@nongnu.org; Wed, 17 May 2023 13:46:50 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.129.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pzLEc-0004PL-SE for qemu-devel@nongnu.org; Wed, 17 May 2023 13:46:49 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1684345605; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=e6kbBe+bveL6T0RzMAZoDW/nord6nL/So0Jk3UEjD+I=; b=T5+Mx4hmI6SXyKOT5Fm8gHo14aC2SxilPbFzgjF3E4tG40hrRaqwl/wBssV9DiwAW4vs3p lGkX6ZAjQIHA77p4SojiQvMkppZbGs6X3dhsxr2jD8KcP55HETyTa4qaJ8TUmgIJ5Y8HMX olM3OBBmhOpkezdmXPevx0/R9El7B34= Received: from mail-wm1-f71.google.com (mail-wm1-f71.google.com [209.85.128.71]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-203-aMwPXeSrPT6CyYNsUTkejg-1; Wed, 17 May 2023 13:46:44 -0400 X-MC-Unique: aMwPXeSrPT6CyYNsUTkejg-1 Received: by mail-wm1-f71.google.com with SMTP id 5b1f17b1804b1-3f4ef4bf00dso6600565e9.1 for ; Wed, 17 May 2023 10:46:43 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1684345602; x=1686937602; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=e6kbBe+bveL6T0RzMAZoDW/nord6nL/So0Jk3UEjD+I=; b=gX0EMEbdDn1hEbDXyqc0dj7urD6QZyVSx4IP/31lp2wLIRUDxwv2xzfdFAhGZSnDYP oXcvGaIj6EcIYfiEAERi7tfzqM0BJO9xFUrinY1i+t1kCFGv9szmskF06meNfd2OikLk 2OUlUm/zPH55RKt0xQxmxZUdUl5QfDLc/2JBS1sXfZM7L9qglTmwyWmIH6i6D3jkidnK LhAJC2nkxkv/Zpf8znncE3jvyWZj7IaKX3GQ82VK58CV9C7xfVclLq7KlVIXE5CYf0bk TFxDmNUH1qQtbcwfu1aLdDXLs8GbUWyocNfUKcPyB78vhBBF13ueFPEaOY4m9A5dWC8a 9mIg== X-Gm-Message-State: AC+VfDwIyHvtYjSHuxLX3IUE/DQhwQ8Rv8Qh2/sbA8jLMoTeVSCbJpP0 J7Ppl/4pe4lBw/q5CJqEwoN6nDomPIEnV6CtVWY4yq+2+HBBVGNnTAS53kRUk+AZnjiOiKrCWRQ rWMiIxdGlLvze+itQs1PMmNxWU5B1Yel5OWjtsp9XOcrqwvK1/aMO0mfU/pTdDJURbmKx44jGQD E= X-Received: by 2002:a7b:cc13:0:b0:3f3:468d:d017 with SMTP id f19-20020a7bcc13000000b003f3468dd017mr32474314wmh.6.1684345602319; Wed, 17 May 2023 10:46:42 -0700 (PDT) X-Google-Smtp-Source: ACHHUZ6RVTbLOCOEFttyRA9a3uecsVTYnmbLXc5TZOCfb3ED6Tg1g2OEIl4YGyCd7GE1IwYKmGkMMQ== X-Received: by 2002:a7b:cc13:0:b0:3f3:468d:d017 with SMTP id f19-20020a7bcc13000000b003f3468dd017mr32474304wmh.6.1684345601749; Wed, 17 May 2023 10:46:41 -0700 (PDT) Received: from [192.168.10.118] ([2001:b07:6468:f312:9af8:e5f5:7516:fa89]) by smtp.gmail.com with ESMTPSA id f21-20020a7bcc15000000b003f31d44f0cbsm2878899wmh.29.2023.05.17.10.46.40 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 17 May 2023 10:46:41 -0700 (PDT) From: Paolo Bonzini To: qemu-devel@nongnu.org Cc: =?utf-8?q?Marc-Andr=C3=A9_Lureau?= Subject: [PULL 48/68] meson: remove static_kwargs Date: Wed, 17 May 2023 19:45:00 +0200 Message-Id: <20230517174520.887405-49-pbonzini@redhat.com> X-Mailer: git-send-email 2.40.1 In-Reply-To: <20230517174520.887405-1-pbonzini@redhat.com> References: <20230517174520.887405-1-pbonzini@redhat.com> MIME-Version: 1.0 Received-SPF: pass client-ip=170.10.129.124; envelope-from=pbonzini@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org After static_kwargs has been changed to an empty dictionary, it has no functional effect and can be removed. Reviewed-by: Marc-AndrĂ© Lureau Signed-off-by: Paolo Bonzini --- meson.build | 215 +++++++++++++++++++----------------------------- tcg/meson.build | 2 +- 2 files changed, 85 insertions(+), 132 deletions(-) diff --git a/meson.build b/meson.build index ac5bb903d2c6..132f309bdab2 100644 --- a/meson.build +++ b/meson.build @@ -18,7 +18,6 @@ targetos = host_machine.system() sh = find_program('sh') config_host = keyval.load(meson.current_build_dir() / 'config-host.mak') enable_modules = 'CONFIG_MODULES' in config_host -static_kwargs = {} cc = meson.get_compiler('c') all_languages = ['c'] @@ -515,7 +514,7 @@ gdbus_codegen = not_found gdbus_codegen_error = '@0@ requires gdbus-codegen, please install libgio' if not get_option('gio').auto() or have_system gio = dependency('gio-2.0', required: get_option('gio'), - method: 'pkg-config', kwargs: static_kwargs) + method: 'pkg-config') if gio.found() and not cc.links(''' #include int main(void) @@ -532,7 +531,7 @@ if not get_option('gio').auto() or have_system gdbus_codegen = find_program(gio.get_variable('gdbus_codegen'), required: get_option('gio')) gio_unix = dependency('gio-unix-2.0', required: get_option('gio'), - method: 'pkg-config', kwargs: static_kwargs) + method: 'pkg-config') gio = declare_dependency(dependencies: [gio, gio_unix], version: gio.version()) endif @@ -545,20 +544,19 @@ endif lttng = not_found if 'ust' in get_option('trace_backends') lttng = dependency('lttng-ust', required: true, version: '>= 2.1', - method: 'pkg-config', kwargs: static_kwargs) + method: 'pkg-config') endif pixman = not_found if have_system or have_tools pixman = dependency('pixman-1', required: have_system, version:'>=0.21.8', - method: 'pkg-config', kwargs: static_kwargs) + method: 'pkg-config') endif -zlib = dependency('zlib', required: true, kwargs: static_kwargs) +zlib = dependency('zlib', required: true) libaio = not_found if not get_option('linux_aio').auto() or have_block libaio = cc.find_library('aio', has_headers: ['libaio.h'], - required: get_option('linux_aio'), - kwargs: static_kwargs) + required: get_option('linux_aio')) endif linux_io_uring_test = ''' @@ -571,7 +569,7 @@ linux_io_uring = not_found if not get_option('linux_io_uring').auto() or have_block linux_io_uring = dependency('liburing', version: '>=0.3', required: get_option('linux_io_uring'), - method: 'pkg-config', kwargs: static_kwargs) + method: 'pkg-config') if not cc.links(linux_io_uring_test) linux_io_uring = not_found endif @@ -581,7 +579,7 @@ libnfs = not_found if not get_option('libnfs').auto() or have_block libnfs = dependency('libnfs', version: '>=1.9.3', required: get_option('libnfs'), - method: 'pkg-config', kwargs: static_kwargs) + method: 'pkg-config') endif libattr_test = ''' @@ -601,8 +599,7 @@ if get_option('attr').allowed() libattr = declare_dependency() else libattr = cc.find_library('attr', has_headers: ['attr/xattr.h'], - required: get_option('attr'), - kwargs: static_kwargs) + required: get_option('attr')) if libattr.found() and not \ cc.links(libattr_test, dependencies: libattr, args: '-DCONFIG_LIBATTR') libattr = not_found @@ -637,7 +634,7 @@ seccomp_has_sysrawrc = false if not get_option('seccomp').auto() or have_system or have_tools seccomp = dependency('libseccomp', version: '>=2.3.0', required: get_option('seccomp'), - method: 'pkg-config', kwargs: static_kwargs) + method: 'pkg-config') if seccomp.found() seccomp_has_sysrawrc = cc.has_header_symbol('seccomp.h', 'SCMP_FLTATR_API_SYSRAWRC', @@ -648,8 +645,7 @@ endif libcap_ng = not_found if not get_option('cap_ng').auto() or have_system or have_tools libcap_ng = cc.find_library('cap-ng', has_headers: ['cap-ng.h'], - required: get_option('cap_ng'), - kwargs: static_kwargs) + required: get_option('cap_ng')) endif if libcap_ng.found() and not cc.links(''' #include @@ -670,13 +666,13 @@ if get_option('xkbcommon').auto() and not have_system and not have_tools xkbcommon = not_found else xkbcommon = dependency('xkbcommon', required: get_option('xkbcommon'), - method: 'pkg-config', kwargs: static_kwargs) + method: 'pkg-config') endif slirp = not_found if not get_option('slirp').auto() or have_system slirp = dependency('slirp', required: get_option('slirp'), - method: 'pkg-config', kwargs: static_kwargs) + method: 'pkg-config') # slirp < 4.7 is incompatible with CFI support in QEMU. This is because # it passes function pointers within libslirp as callbacks for timers. # When using a system-wide shared libslirp, the type information for the @@ -696,8 +692,7 @@ endif vde = not_found if not get_option('vde').auto() or have_system or have_tools vde = cc.find_library('vdeplug', has_headers: ['libvdeplug.h'], - required: get_option('vde'), - kwargs: static_kwargs) + required: get_option('vde')) endif if vde.found() and not cc.links(''' #include @@ -719,41 +714,41 @@ endif pulse = not_found if not get_option('pa').auto() or (targetos == 'linux' and have_system) pulse = dependency('libpulse', required: get_option('pa'), - method: 'pkg-config', kwargs: static_kwargs) + method: 'pkg-config') endif alsa = not_found if not get_option('alsa').auto() or (targetos == 'linux' and have_system) alsa = dependency('alsa', required: get_option('alsa'), - method: 'pkg-config', kwargs: static_kwargs) + method: 'pkg-config') endif jack = not_found if not get_option('jack').auto() or have_system jack = dependency('jack', required: get_option('jack'), - method: 'pkg-config', kwargs: static_kwargs) + method: 'pkg-config') endif pipewire = not_found if not get_option('pipewire').auto() or (targetos == 'linux' and have_system) pipewire = dependency('libpipewire-0.3', version: '>=0.3.60', required: get_option('pipewire'), - method: 'pkg-config', kwargs: static_kwargs) + method: 'pkg-config') endif sndio = not_found if not get_option('sndio').auto() or have_system sndio = dependency('sndio', required: get_option('sndio'), - method: 'pkg-config', kwargs: static_kwargs) + method: 'pkg-config') endif spice_protocol = not_found if not get_option('spice_protocol').auto() or have_system spice_protocol = dependency('spice-protocol', version: '>=0.14.0', required: get_option('spice_protocol'), - method: 'pkg-config', kwargs: static_kwargs) + method: 'pkg-config') endif spice = not_found if not get_option('spice').auto() or have_system spice = dependency('spice-server', version: '>=0.14.0', required: get_option('spice'), - method: 'pkg-config', kwargs: static_kwargs) + method: 'pkg-config') endif spice_headers = spice.partial_dependency(compile_args: true, includes: true) @@ -763,13 +758,13 @@ libiscsi = not_found if not get_option('libiscsi').auto() or have_block libiscsi = dependency('libiscsi', version: '>=1.9.0', required: get_option('libiscsi'), - method: 'pkg-config', kwargs: static_kwargs) + method: 'pkg-config') endif zstd = not_found if not get_option('zstd').auto() or have_block zstd = dependency('libzstd', version: '>=1.4.0', required: get_option('zstd'), - method: 'pkg-config', kwargs: static_kwargs) + method: 'pkg-config') endif virgl = not_found @@ -777,29 +772,25 @@ have_vhost_user_gpu = have_tools and targetos == 'linux' and pixman.found() if not get_option('virglrenderer').auto() or have_system or have_vhost_user_gpu virgl = dependency('virglrenderer', method: 'pkg-config', - required: get_option('virglrenderer'), - kwargs: static_kwargs) + required: get_option('virglrenderer')) endif blkio = not_found if not get_option('blkio').auto() or have_block blkio = dependency('blkio', method: 'pkg-config', - required: get_option('blkio'), - kwargs: static_kwargs) + required: get_option('blkio')) endif curl = not_found if not get_option('curl').auto() or have_block curl = dependency('libcurl', version: '>=7.29.0', method: 'pkg-config', - required: get_option('curl'), - kwargs: static_kwargs) + required: get_option('curl')) endif libudev = not_found if targetos == 'linux' and (have_system or have_tools) libudev = dependency('libudev', method: 'pkg-config', - required: get_option('libudev'), - kwargs: static_kwargs) + required: get_option('libudev')) endif mpathlibs = [libudev] @@ -834,18 +825,15 @@ if targetos == 'linux' and have_tools and get_option('mpath').allowed() return 0; }''' libmpathpersist = cc.find_library('mpathpersist', - required: get_option('mpath'), - kwargs: static_kwargs) + required: get_option('mpath')) if libmpathpersist.found() mpathlibs += libmpathpersist if get_option('prefer_static') mpathlibs += cc.find_library('devmapper', - required: get_option('mpath'), - kwargs: static_kwargs) + required: get_option('mpath')) endif mpathlibs += cc.find_library('multipath', - required: get_option('mpath'), - kwargs: static_kwargs) + required: get_option('mpath')) foreach lib: mpathlibs if not lib.found() mpathlibs = [] @@ -895,8 +883,7 @@ if have_system and get_option('curses').allowed() curses_dep_list = targetos == 'windows' ? ['ncurses', 'ncursesw'] : ['ncursesw'] curses = dependency(curses_dep_list, required: false, - method: 'pkg-config', - kwargs: static_kwargs) + method: 'pkg-config') msg = get_option('curses').enabled() ? 'curses library not found' : '' curses_compile_args = ['-DNCURSES_WIDECHAR=1'] if curses.found() @@ -918,8 +905,7 @@ if have_system and get_option('curses').allowed() curses_libname_list = (targetos == 'windows' ? ['pdcurses'] : ['ncursesw', 'cursesw']) foreach curses_libname : curses_libname_list libcurses = cc.find_library(curses_libname, - required: false, - kwargs: static_kwargs) + required: false) if libcurses.found() if cc.links(curses_test, args: curses_compile_args, dependencies: libcurses) curses = declare_dependency(compile_args: curses_compile_args, @@ -969,8 +955,7 @@ endif brlapi = not_found if not get_option('brlapi').auto() or have_system brlapi = cc.find_library('brlapi', has_headers: ['brlapi.h'], - required: get_option('brlapi'), - kwargs: static_kwargs) + required: get_option('brlapi')) if brlapi.found() and not cc.links(''' #include #include @@ -986,7 +971,7 @@ endif sdl = not_found if not get_option('sdl').auto() or have_system - sdl = dependency('sdl2', required: get_option('sdl'), kwargs: static_kwargs) + sdl = dependency('sdl2', required: get_option('sdl')) sdl_image = not_found endif if sdl.found() @@ -994,7 +979,7 @@ if sdl.found() sdl = declare_dependency(compile_args: '-Wno-undef', dependencies: sdl) sdl_image = dependency('SDL2_image', required: get_option('sdl_image'), - method: 'pkg-config', kwargs: static_kwargs) + method: 'pkg-config') else if get_option('sdl_image').enabled() error('sdl-image required, but SDL was @0@'.format( @@ -1005,11 +990,9 @@ endif rbd = not_found if not get_option('rbd').auto() or have_block - librados = cc.find_library('rados', required: get_option('rbd'), - kwargs: static_kwargs) + librados = cc.find_library('rados', required: get_option('rbd')) librbd = cc.find_library('rbd', has_headers: ['rbd/librbd.h'], - required: get_option('rbd'), - kwargs: static_kwargs) + required: get_option('rbd')) if librados.found() and librbd.found() if cc.links(''' #include @@ -1037,7 +1020,7 @@ glusterfs_iocb_has_stat = false if not get_option('glusterfs').auto() or have_block glusterfs = dependency('glusterfs-api', version: '>=3', required: get_option('glusterfs'), - method: 'pkg-config', kwargs: static_kwargs) + method: 'pkg-config') if glusterfs.found() glusterfs_ftruncate_has_stat = cc.links(''' #include @@ -1072,15 +1055,13 @@ libssh = not_found if not get_option('libssh').auto() or have_block libssh = dependency('libssh', version: '>=0.8.7', method: 'pkg-config', - required: get_option('libssh'), - kwargs: static_kwargs) + required: get_option('libssh')) endif libbzip2 = not_found if not get_option('bzip2').auto() or have_block libbzip2 = cc.find_library('bz2', has_headers: ['bzlib.h'], - required: get_option('bzip2'), - kwargs: static_kwargs) + required: get_option('bzip2')) if libbzip2.found() and not cc.links(''' #include int main(void) { BZ2_bzlibVersion(); return 0; }''', dependencies: libbzip2) @@ -1096,8 +1077,7 @@ endif liblzfse = not_found if not get_option('lzfse').auto() or have_block liblzfse = cc.find_library('lzfse', has_headers: ['lzfse.h'], - required: get_option('lzfse'), - kwargs: static_kwargs) + required: get_option('lzfse')) endif if liblzfse.found() and not cc.links(''' #include @@ -1115,8 +1095,7 @@ if get_option('oss').allowed() and have_system if not cc.has_header('sys/soundcard.h') # not found elif targetos == 'netbsd' - oss = cc.find_library('ossaudio', required: get_option('oss'), - kwargs: static_kwargs) + oss = cc.find_library('ossaudio', required: get_option('oss')) else oss = declare_dependency() endif @@ -1149,7 +1128,7 @@ endif opengl = not_found if not get_option('opengl').auto() or have_system or have_vhost_user_gpu epoxy = dependency('epoxy', method: 'pkg-config', - required: get_option('opengl'), kwargs: static_kwargs) + required: get_option('opengl')) if cc.has_header('epoxy/egl.h', dependencies: epoxy) opengl = epoxy elif get_option('opengl').enabled() @@ -1158,8 +1137,7 @@ if not get_option('opengl').auto() or have_system or have_vhost_user_gpu endif gbm = not_found if (have_system or have_tools) and (virgl.found() or opengl.found()) - gbm = dependency('gbm', method: 'pkg-config', required: false, - kwargs: static_kwargs) + gbm = dependency('gbm', method: 'pkg-config', required: false) endif have_vhost_user_gpu = have_vhost_user_gpu and virgl.found() and opengl.found() and gbm.found() @@ -1181,16 +1159,14 @@ if get_option('gnutls').enabled() or (get_option('gnutls').auto() and have_syste # the platform support requirements gnutls_crypto = dependency('gnutls', version: '>=3.6.14', method: 'pkg-config', - required: false, - kwargs: static_kwargs) + required: false) if gnutls_crypto.found() gnutls = gnutls_crypto else # Our min version if all we need is TLS gnutls = dependency('gnutls', version: '>=3.5.18', method: 'pkg-config', - required: get_option('gnutls'), - kwargs: static_kwargs) + required: get_option('gnutls')) endif endif @@ -1217,34 +1193,31 @@ if not gnutls_crypto.found() if (not get_option('gcrypt').auto() or have_system) and not get_option('nettle').enabled() gcrypt = dependency('libgcrypt', version: '>=1.8', method: 'config-tool', - required: get_option('gcrypt'), - kwargs: static_kwargs) + required: get_option('gcrypt')) # Debian has removed -lgpg-error from libgcrypt-config # as it "spreads unnecessary dependencies" which in # turn breaks static builds... if gcrypt.found() and get_option('prefer_static') gcrypt = declare_dependency(dependencies: [ gcrypt, - cc.find_library('gpg-error', required: true, kwargs: static_kwargs)]) + cc.find_library('gpg-error', required: true)]) endif endif if (not get_option('nettle').auto() or have_system) and not gcrypt.found() nettle = dependency('nettle', version: '>=3.4', method: 'pkg-config', - required: get_option('nettle'), - kwargs: static_kwargs) + required: get_option('nettle')) if nettle.found() and not cc.has_header('nettle/xts.h', dependencies: nettle) xts = 'private' endif endif endif -gmp = dependency('gmp', required: false, method: 'pkg-config', kwargs: static_kwargs) +gmp = dependency('gmp', required: false, method: 'pkg-config') if nettle.found() and gmp.found() hogweed = dependency('hogweed', version: '>=3.4', method: 'pkg-config', - required: get_option('nettle'), - kwargs: static_kwargs) + required: get_option('nettle')) endif @@ -1256,20 +1229,17 @@ have_gtk_clipboard = get_option('gtk_clipboard').enabled() if not get_option('gtk').auto() or have_system gtk = dependency('gtk+-3.0', version: '>=3.22.0', method: 'pkg-config', - required: get_option('gtk'), - kwargs: static_kwargs) + required: get_option('gtk')) if gtk.found() gtkx11 = dependency('gtk+-x11-3.0', version: '>=3.22.0', method: 'pkg-config', - required: false, - kwargs: static_kwargs) + required: false) gtk = declare_dependency(dependencies: [gtk, gtkx11]) if not get_option('vte').auto() or have_system vte = dependency('vte-2.91', method: 'pkg-config', - required: get_option('vte'), - kwargs: static_kwargs) + required: get_option('vte')) endif elif have_gtk_clipboard error('GTK clipboard requested, but GTK not found') @@ -1278,13 +1248,12 @@ endif x11 = not_found if gtkx11.found() - x11 = dependency('x11', method: 'pkg-config', required: gtkx11.found(), - kwargs: static_kwargs) + x11 = dependency('x11', method: 'pkg-config', required: gtkx11.found()) endif png = not_found if get_option('png').allowed() and have_system png = dependency('libpng', version: '>=1.6.34', required: get_option('png'), - method: 'pkg-config', kwargs: static_kwargs) + method: 'pkg-config') endif vnc = not_found jpeg = not_found @@ -1292,10 +1261,9 @@ sasl = not_found if get_option('vnc').allowed() and have_system vnc = declare_dependency() # dummy dependency jpeg = dependency('libjpeg', required: get_option('vnc_jpeg'), - method: 'pkg-config', kwargs: static_kwargs) + method: 'pkg-config') sasl = cc.find_library('sasl2', has_headers: ['sasl/sasl.h'], - required: get_option('vnc_sasl'), - kwargs: static_kwargs) + required: get_option('vnc_sasl')) if sasl.found() sasl = declare_dependency(dependencies: sasl, compile_args: '-DSTRUCT_IOVEC_DEFINED') @@ -1305,8 +1273,7 @@ endif pam = not_found if not get_option('auth_pam').auto() or have_system pam = cc.find_library('pam', has_headers: ['security/pam_appl.h'], - required: get_option('auth_pam'), - kwargs: static_kwargs) + required: get_option('auth_pam')) endif if pam.found() and not cc.links(''' #include @@ -1330,8 +1297,7 @@ endif snappy = not_found if not get_option('snappy').auto() or have_system snappy = cc.find_library('snappy', has_headers: ['snappy-c.h'], - required: get_option('snappy'), - kwargs: static_kwargs) + required: get_option('snappy')) endif if snappy.found() and not linker.links(''' #include @@ -1347,8 +1313,7 @@ endif lzo = not_found if not get_option('lzo').auto() or have_system lzo = cc.find_library('lzo2', has_headers: ['lzo/lzo1x.h'], - required: get_option('lzo'), - kwargs: static_kwargs) + required: get_option('lzo')) endif if lzo.found() and not cc.links(''' #include @@ -1364,8 +1329,7 @@ endif numa = not_found if not get_option('numa').auto() or have_system or have_tools numa = cc.find_library('numa', has_headers: ['numa.h'], - required: get_option('numa'), - kwargs: static_kwargs) + required: get_option('numa')) endif if numa.found() and not cc.links(''' #include @@ -1383,10 +1347,8 @@ rdma = not_found if not get_option('rdma').auto() or have_system libumad = cc.find_library('ibumad', required: get_option('rdma')) rdma_libs = [cc.find_library('rdmacm', has_headers: ['rdma/rdma_cma.h'], - required: get_option('rdma'), - kwargs: static_kwargs), - cc.find_library('ibverbs', required: get_option('rdma'), - kwargs: static_kwargs), + required: get_option('rdma')), + cc.find_library('ibverbs', required: get_option('rdma')), libumad] rdma = declare_dependency(dependencies: rdma_libs) foreach lib: rdma_libs @@ -1399,30 +1361,30 @@ endif xen = not_found if get_option('xen').enabled() or (get_option('xen').auto() and have_system) xencontrol = dependency('xencontrol', required: false, - method: 'pkg-config', kwargs: static_kwargs) + method: 'pkg-config') if xencontrol.found() xen_pc = declare_dependency(version: xencontrol.version(), dependencies: [ xencontrol, # disabler: true makes xen_pc.found() return false if any is not found dependency('xenstore', required: false, - method: 'pkg-config', kwargs: static_kwargs, + method: 'pkg-config', disabler: true), dependency('xenforeignmemory', required: false, - method: 'pkg-config', kwargs: static_kwargs, + method: 'pkg-config', disabler: true), dependency('xengnttab', required: false, - method: 'pkg-config', kwargs: static_kwargs, + method: 'pkg-config', disabler: true), dependency('xenevtchn', required: false, - method: 'pkg-config', kwargs: static_kwargs, + method: 'pkg-config', disabler: true), dependency('xendevicemodel', required: false, - method: 'pkg-config', kwargs: static_kwargs, + method: 'pkg-config', disabler: true), # optional, no "disabler: true" dependency('xentoolcore', required: false, - method: 'pkg-config', kwargs: static_kwargs)]) + method: 'pkg-config')]) if xen_pc.found() xen = xen_pc endif @@ -1480,60 +1442,53 @@ have_xen_pci_passthrough = get_option('xen_pci_passthrough') \ cacard = not_found if not get_option('smartcard').auto() or have_system cacard = dependency('libcacard', required: get_option('smartcard'), - version: '>=2.5.1', method: 'pkg-config', - kwargs: static_kwargs) + version: '>=2.5.1', method: 'pkg-config') endif u2f = not_found if have_system u2f = dependency('u2f-emu', required: get_option('u2f'), - method: 'pkg-config', - kwargs: static_kwargs) + method: 'pkg-config') endif canokey = not_found if have_system canokey = dependency('canokey-qemu', required: get_option('canokey'), - method: 'pkg-config', - kwargs: static_kwargs) + method: 'pkg-config') endif usbredir = not_found if not get_option('usb_redir').auto() or have_system usbredir = dependency('libusbredirparser-0.5', required: get_option('usb_redir'), - version: '>=0.6', method: 'pkg-config', - kwargs: static_kwargs) + version: '>=0.6', method: 'pkg-config') endif libusb = not_found if not get_option('libusb').auto() or have_system libusb = dependency('libusb-1.0', required: get_option('libusb'), - version: '>=1.0.13', method: 'pkg-config', - kwargs: static_kwargs) + version: '>=1.0.13', method: 'pkg-config') endif libpmem = not_found if not get_option('libpmem').auto() or have_system libpmem = dependency('libpmem', required: get_option('libpmem'), - method: 'pkg-config', kwargs: static_kwargs) + method: 'pkg-config') endif libdaxctl = not_found if not get_option('libdaxctl').auto() or have_system libdaxctl = dependency('libdaxctl', required: get_option('libdaxctl'), - version: '>=57', method: 'pkg-config', - kwargs: static_kwargs) + version: '>=57', method: 'pkg-config') endif tasn1 = not_found if gnutls.found() tasn1 = dependency('libtasn1', - method: 'pkg-config', - kwargs: static_kwargs) + method: 'pkg-config') endif keyutils = dependency('libkeyutils', required: false, - method: 'pkg-config', kwargs: static_kwargs) + method: 'pkg-config') has_gettid = cc.has_function('gettid') # libselinux selinux = dependency('libselinux', required: get_option('selinux'), - method: 'pkg-config', kwargs: static_kwargs) + method: 'pkg-config') # Malloc tests @@ -1597,8 +1552,7 @@ if get_option('fuse').disabled() and get_option('fuse_lseek').enabled() endif fuse = dependency('fuse3', required: get_option('fuse'), - version: '>=3.1', method: 'pkg-config', - kwargs: static_kwargs) + version: '>=3.1', method: 'pkg-config') fuse_lseek = not_found if get_option('fuse_lseek').allowed() @@ -1657,7 +1611,6 @@ if not get_option('libdw').auto() or \ (not get_option('prefer_static') and (have_system or have_user)) libdw = dependency('libdw', method: 'pkg-config', - kwargs: static_kwargs, required: get_option('libdw')) endif @@ -2737,7 +2690,7 @@ genh += custom_target('config-poison.h', capstone = not_found if not get_option('capstone').auto() or have_system or have_user capstone = dependency('capstone', version: '>=3.0.5', - kwargs: static_kwargs, method: 'pkg-config', + method: 'pkg-config', required: get_option('capstone')) # Some versions of capstone have broken pkg-config file @@ -2773,7 +2726,7 @@ if have_system fdt_opt = get_option('fdt') if fdt_opt in ['enabled', 'auto', 'system'] have_internal = fs.exists(meson.current_source_dir() / 'dtc/libfdt/Makefile.libfdt') - fdt = cc.find_library('fdt', kwargs: static_kwargs, + fdt = cc.find_library('fdt', required: fdt_opt == 'system' or fdt_opt == 'enabled' and not have_internal) if fdt.found() and cc.links(''' diff --git a/tcg/meson.build b/tcg/meson.build index c4c63b19d4ed..3db551bb3485 100644 --- a/tcg/meson.build +++ b/tcg/meson.build @@ -12,7 +12,7 @@ tcg_ss.add(files( if get_option('tcg_interpreter') libffi = dependency('libffi', version: '>=3.0', required: true, - method: 'pkg-config', kwargs: static_kwargs) + method: 'pkg-config') specific_ss.add(libffi) specific_ss.add(files('tci.c')) endif From patchwork Wed May 17 17:45:01 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Paolo Bonzini X-Patchwork-Id: 1782884 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=nongnu.org (client-ip=209.51.188.17; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=) Authentication-Results: legolas.ozlabs.org; dkim=pass (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=dROU/QQo; dkim-atps=neutral Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-ECDSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4QM1FG4MmMz20dn for ; Thu, 18 May 2023 04:03:46 +1000 (AEST) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1pzLH0-00057j-9x; Wed, 17 May 2023 13:49:14 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pzLEf-0001bR-4y for qemu-devel@nongnu.org; Wed, 17 May 2023 13:46:49 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pzLEc-0004PT-De for qemu-devel@nongnu.org; Wed, 17 May 2023 13:46:48 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1684345605; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=BTl8dz4S4KQFdMauiBtgxB/zEVjX+4oDG4c1HH7MeJg=; b=dROU/QQo/5Gqs+/SoI6CQDHdwQRg3vlDnGZU/M/WCMK2a7kKWFXrFimy4qZQgwnrQeZnw0 C0QohA/BQyxFVCJXnxZyAYBFrBiq+r5e5/nTQtkI+Es/c1nCwp2mtAtOB/HvIrzw656xZP vv62RF0GoaX2Q3MYsEtlcLpbq8WWt3Y= Received: from mail-wr1-f72.google.com (mail-wr1-f72.google.com [209.85.221.72]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-178-qRe7LmtVPNGxYTivGFKGww-1; Wed, 17 May 2023 13:46:44 -0400 X-MC-Unique: qRe7LmtVPNGxYTivGFKGww-1 Received: by mail-wr1-f72.google.com with SMTP id ffacd0b85a97d-3063089cff4so679066f8f.0 for ; Wed, 17 May 2023 10:46:44 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1684345603; x=1686937603; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=BTl8dz4S4KQFdMauiBtgxB/zEVjX+4oDG4c1HH7MeJg=; b=ZCSDDiISSkwAGmpJR81ovRK8tYOrch/D2xGJ5JqBE5IYwIm2wFK6F4PE5Eo51vwq+v qzugMaRellKqQSDaFzmmfZII/wn7OYqLIjkQwjzq81Q33BwmJ7WfP6UL+zWMeHlU+Mdy Ela/zPvNTJ9yB7kowhzs+3OIH11BkfLki5bjFMQvj9tlofLkOnSKFln6Txcf/cVcFr7d CqpkFFmbz0WTG9IQoBcGJCjeWc47ncI5CtGDRX4P2qyjhKRY1KLs2CvZ0nbNS4Lj+BOc iuWfSaocX9c+8f3re1hmBhZjUWji5wRNmmQISBSwD0sIZwC284bODthRRs6u+9eOxhmm pv3A== X-Gm-Message-State: AC+VfDxd5o+Mul97EZjaYjjql6Tap+VJWGeq1UKjeC67K3ZKC4+uIa+u yAFiol963fIU9L0g97uQGyy6Ze0LEBw/ni3Y5XDbayK6d/rPbS+f3HSCIdSW6sIxRMpduyJ8ZJg 9yK65l2cFeOrSPWIrEnuHp3TN2j9uTvyO9zmJj1JPBMBimH+SUyigMCMRkeBdsT7CUht+3qJAtq g= X-Received: by 2002:adf:df07:0:b0:307:34d4:7ec8 with SMTP id y7-20020adfdf07000000b0030734d47ec8mr1345097wrl.34.1684345603229; Wed, 17 May 2023 10:46:43 -0700 (PDT) X-Google-Smtp-Source: ACHHUZ6gsIsnbUfO0XtbmZZJeYW+9/FbDq+OSTDmVx2Y3PM/3426MOo0jN3/Y2tnPUWCYcXyUyTtqA== X-Received: by 2002:adf:df07:0:b0:307:34d4:7ec8 with SMTP id y7-20020adfdf07000000b0030734d47ec8mr1345084wrl.34.1684345602894; Wed, 17 May 2023 10:46:42 -0700 (PDT) Received: from [192.168.10.118] ([2001:b07:6468:f312:9af8:e5f5:7516:fa89]) by smtp.gmail.com with ESMTPSA id o4-20020a5d4a84000000b003068f5cca8csm3386942wrq.94.2023.05.17.10.46.42 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 17 May 2023 10:46:42 -0700 (PDT) From: Paolo Bonzini To: qemu-devel@nongnu.org Cc: =?utf-8?q?Philippe_Mathieu-Daud=C3=A9?= , Richard Henderson Subject: [PULL 49/68] meson: add more version numbers to the summary Date: Wed, 17 May 2023 19:45:01 +0200 Message-Id: <20230517174520.887405-50-pbonzini@redhat.com> X-Mailer: git-send-email 2.40.1 In-Reply-To: <20230517174520.887405-1-pbonzini@redhat.com> References: <20230517174520.887405-1-pbonzini@redhat.com> MIME-Version: 1.0 Received-SPF: pass client-ip=170.10.133.124; envelope-from=pbonzini@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Whenever declare_dependency is used to add some compile flags or dependent libraries to the outcome of dependency(), the version of the original dependency is dropped in the summary. Make sure that declare_dependency() has a version argument in those cases. Reviewed-by: Philippe Mathieu-DaudĂ© Reviewed-by: Richard Henderson Signed-off-by: Paolo Bonzini --- meson.build | 16 ++++++++++------ 1 file changed, 10 insertions(+), 6 deletions(-) diff --git a/meson.build b/meson.build index 132f309bdab2..464890d99ac0 100644 --- a/meson.build +++ b/meson.build @@ -888,7 +888,8 @@ if have_system and get_option('curses').allowed() curses_compile_args = ['-DNCURSES_WIDECHAR=1'] if curses.found() if cc.links(curses_test, args: curses_compile_args, dependencies: [curses]) - curses = declare_dependency(compile_args: curses_compile_args, dependencies: [curses]) + curses = declare_dependency(compile_args: curses_compile_args, dependencies: [curses], + version: curses.version()) else msg = 'curses package not usable' curses = not_found @@ -977,7 +978,8 @@ endif if sdl.found() # work around 2.0.8 bug sdl = declare_dependency(compile_args: '-Wno-undef', - dependencies: sdl) + dependencies: sdl, + version: sdl.version()) sdl_image = dependency('SDL2_image', required: get_option('sdl_image'), method: 'pkg-config') else @@ -1198,9 +1200,10 @@ if not gnutls_crypto.found() # as it "spreads unnecessary dependencies" which in # turn breaks static builds... if gcrypt.found() and get_option('prefer_static') - gcrypt = declare_dependency(dependencies: [ - gcrypt, - cc.find_library('gpg-error', required: true)]) + gcrypt = declare_dependency(dependencies: + [gcrypt, + cc.find_library('gpg-error', required: true)], + version: gcrypt.version()) endif endif if (not get_option('nettle').auto() or have_system) and not gcrypt.found() @@ -1234,7 +1237,8 @@ if not get_option('gtk').auto() or have_system gtkx11 = dependency('gtk+-x11-3.0', version: '>=3.22.0', method: 'pkg-config', required: false) - gtk = declare_dependency(dependencies: [gtk, gtkx11]) + gtk = declare_dependency(dependencies: [gtk, gtkx11], + version: gtk.version()) if not get_option('vte').auto() or have_system vte = dependency('vte-2.91', From patchwork Wed May 17 17:45:02 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Paolo Bonzini X-Patchwork-Id: 1782890 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=nongnu.org (client-ip=209.51.188.17; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=) Authentication-Results: legolas.ozlabs.org; dkim=pass (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=TwXO5fZf; dkim-atps=neutral Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-ECDSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4QM1G76kLWz20dn for ; Thu, 18 May 2023 04:04:31 +1000 (AEST) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1pzLH0-00058y-L1; Wed, 17 May 2023 13:49:15 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pzLEg-0001d9-BP for qemu-devel@nongnu.org; Wed, 17 May 2023 13:46:50 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.129.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pzLEe-0004Ps-Ij for qemu-devel@nongnu.org; Wed, 17 May 2023 13:46:49 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1684345608; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=mG9RV/wEzHBntPeSF5MlucOTxAQIk0J7mP2S/5XHYJY=; b=TwXO5fZfZOCf5HtxhTYtmk3K7F5TRaHDWtHU66AixDrgLeRlB9AEYjVOFkA0Wr4t+mFk2q K+qCoT8E3aR+z7vZdY1SgIk/4YUyfxpZq7mD5cS1h1COjoG9qgJ5Axu5xGnCR3DePu9jp/ dGR+fAOD52C3+G46qJHdMeVTJcZpo30= Received: from mail-wm1-f70.google.com (mail-wm1-f70.google.com [209.85.128.70]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-93-32UQ6sgCNP-TFDYWwV0BgQ-1; Wed, 17 May 2023 13:46:46 -0400 X-MC-Unique: 32UQ6sgCNP-TFDYWwV0BgQ-1 Received: by mail-wm1-f70.google.com with SMTP id 5b1f17b1804b1-3f426514683so4396355e9.1 for ; Wed, 17 May 2023 10:46:46 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1684345605; x=1686937605; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=mG9RV/wEzHBntPeSF5MlucOTxAQIk0J7mP2S/5XHYJY=; b=UsIJm9DnlHkpIb3173/QYBtpOs28wiWWpEjhVgBDqa8IQzLgO8RTV5OEV/cCDO6M7Y Lyv3LVH29hGAk+yX2n/9tu0myK7gWH2k6HRgUFlaBeQO6eYWyBBubW6DJkIOv7OnrFhb 0XU+pHP1r806KWpKL6UTOU9BysJ/mPvAJVQFqSz1Gbl5N/Csfv2fhnbQVdpPsRmt2o/N 5Ph+e0L3zfkOWyvGG17si6vftwnxEJGN8Fciaix3vStsMGbhSh6n9E4BzoXiLsw62kDx C7L9buiFBgOTtmY/Fj49vYs7etNwvbBBPEkl6bJhYTHOCtN2+2Ig4onQ0u3Rcdp0Fkso LSgg== X-Gm-Message-State: AC+VfDx6aZ8iVJefo5hUK9V0/swZfPUVVkCDRCEgBOyoaJXp5vb6z2kx BO4TEbvkq7KnWcRysoqYNNcTBP/qbkEOh7eGCGG69auGN3ONnGwtta4oK+NbGaAupRIeeTLAbrA +UGS/OvNnzbRjRvZRwSdy0sSdikrjJphtcciV9YrwKYC56NfCDdPGU/vILPhOcuF9pSR7C7rTad s= X-Received: by 2002:a05:600c:2305:b0:3f5:d3c6:737d with SMTP id 5-20020a05600c230500b003f5d3c6737dmr837810wmo.39.1684345604792; Wed, 17 May 2023 10:46:44 -0700 (PDT) X-Google-Smtp-Source: ACHHUZ6c55JgDR6IXneH1h0+ccVOIRHjDFr8ZIzCvs5LNfVJXZak9kb/JxXnY+GCfOKABZGwxHUlAQ== X-Received: by 2002:a05:600c:2305:b0:3f5:d3c6:737d with SMTP id 5-20020a05600c230500b003f5d3c6737dmr837800wmo.39.1684345604386; Wed, 17 May 2023 10:46:44 -0700 (PDT) Received: from [192.168.10.118] ([2001:b07:6468:f312:9af8:e5f5:7516:fa89]) by smtp.gmail.com with ESMTPSA id f6-20020a1c6a06000000b003f427cba193sm2812575wmc.41.2023.05.17.10.46.43 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 17 May 2023 10:46:43 -0700 (PDT) From: Paolo Bonzini To: qemu-devel@nongnu.org Subject: [PULL 50/68] meson: drop unnecessary declare_dependency() Date: Wed, 17 May 2023 19:45:02 +0200 Message-Id: <20230517174520.887405-51-pbonzini@redhat.com> X-Mailer: git-send-email 2.40.1 In-Reply-To: <20230517174520.887405-1-pbonzini@redhat.com> References: <20230517174520.887405-1-pbonzini@redhat.com> MIME-Version: 1.0 Received-SPF: pass client-ip=170.10.129.124; envelope-from=pbonzini@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org The libvfio_user_dep variable of subprojects/libvfio-user/lib/meson.build is already a dependency, so there is no need to wrap it with another declare_dependency(). Signed-off-by: Paolo Bonzini --- meson.build | 4 +--- 1 file changed, 1 insertion(+), 3 deletions(-) diff --git a/meson.build b/meson.build index 464890d99ac0..88fba57cca81 100644 --- a/meson.build +++ b/meson.build @@ -2720,9 +2720,7 @@ if have_system and vfio_user_server_allowed libvfio_user_proj = subproject('libvfio-user') - libvfio_user_lib = libvfio_user_proj.get_variable('libvfio_user_dep') - - libvfio_user_dep = declare_dependency(dependencies: [libvfio_user_lib]) + libvfio_user_dep = libvfio_user_proj.get_variable('libvfio_user_dep') endif fdt = not_found From patchwork Wed May 17 17:45:03 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Paolo Bonzini X-Patchwork-Id: 1782888 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=nongnu.org (client-ip=209.51.188.17; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=) Authentication-Results: legolas.ozlabs.org; dkim=pass (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=Ridqkh8x; dkim-atps=neutral Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-ECDSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4QM1FM1W6sz20dn for ; Thu, 18 May 2023 04:03:51 +1000 (AEST) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1pzLH9-0005Z8-QA; Wed, 17 May 2023 13:49:23 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pzLEj-0001qN-Ul for qemu-devel@nongnu.org; Wed, 17 May 2023 13:47:00 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.129.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pzLEg-0004QU-Mx for qemu-devel@nongnu.org; Wed, 17 May 2023 13:46:52 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1684345610; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=KGOKOonwVCauOBP4yazjtIQ3TtYGjPeXxWLdCc2VYMM=; b=Ridqkh8xP/wExVM2Z80sZzuv430ikZlMym0Y+siJZHlstKzgg5KEZ9Dyim0CxlxdT84z30 SuDBhvZ3EpHQu49pD5ztGpyFHbHRc3nyCGz9u+q0WWMaFNBOL7RJhOkJOjkz/eb3t5T6e9 nqV0nULPnM4pdr5UdRIoHkEjOHgrCyU= Received: from mail-wm1-f69.google.com (mail-wm1-f69.google.com [209.85.128.69]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-381-hA0zIM4fO_iTp8uJD0FDzw-1; Wed, 17 May 2023 13:46:48 -0400 X-MC-Unique: hA0zIM4fO_iTp8uJD0FDzw-1 Received: by mail-wm1-f69.google.com with SMTP id 5b1f17b1804b1-3f420742d40so4453355e9.2 for ; Wed, 17 May 2023 10:46:48 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1684345606; x=1686937606; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=KGOKOonwVCauOBP4yazjtIQ3TtYGjPeXxWLdCc2VYMM=; b=dsB0ETu7PIgu29ZfrufUCAwgRjagCNmpc84pa+7/g8eXv1BHm/je+9q9z+WzTOfCdE dzKVh+eOLSXYtZggvyQw5hQ75vo/RIu5eJUPqNqQRittyGmkbakS5dT9kh85Wxm1LCz2 Sa+VahpaiHHrU2u17UzZolpv5hItWYg451neVUfHIrQfJbVQdKrqw0q7kafIBEnuFI0l k+m3daK/juRAluIVpU6o+LLZ9Q7rfFr3GJXryUtHBKXfX6CwC/tVbyBTA56s7T4MRcox LsMOe+iPbxr7Bi5bru5GtjDM+aikvQVHMNpQIRrTrhckXignq5yqNTJe9ntRWJAOJ1u/ /l/Q== X-Gm-Message-State: AC+VfDwdaLzZD90SrHe14it2tyaci+qMqBBriiL86+2UDu6WTqQ8mo/F p0kbL5JNEN9nSjO3ZktHgCD4xMi7rFeK7AQXdSvVFLi961nVaswqXuYkJ69SXXC0cNsdBBkC8tR aKvEyl1EwmOdDZQZ+jTq4oW74Q5YF9MEHFEf/xTJA0XJv7W/B3FzwjS86O7VQz4svG6M1W0myXq Y= X-Received: by 2002:a05:600c:231a:b0:3ed:5d41:f95c with SMTP id 26-20020a05600c231a00b003ed5d41f95cmr29731743wmo.11.1684345606386; Wed, 17 May 2023 10:46:46 -0700 (PDT) X-Google-Smtp-Source: ACHHUZ7ADwz1p1flpKBGeqjG6mHuSPFKDLRO6XQvx1xAMFckD1Xr8rHW1XHCbOyCi8NUn2TncpDfbg== X-Received: by 2002:a05:600c:231a:b0:3ed:5d41:f95c with SMTP id 26-20020a05600c231a00b003ed5d41f95cmr29731728wmo.11.1684345605864; Wed, 17 May 2023 10:46:45 -0700 (PDT) Received: from [192.168.10.118] ([2001:b07:6468:f312:9af8:e5f5:7516:fa89]) by smtp.gmail.com with ESMTPSA id l5-20020a5d4105000000b00307b5376b2csm3545299wrp.90.2023.05.17.10.46.44 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 17 May 2023 10:46:45 -0700 (PDT) From: Paolo Bonzini To: qemu-devel@nongnu.org Cc: =?utf-8?q?Philippe_Mathieu-Daud=C3=A9?= Subject: [PULL 51/68] build: move glib detection and workarounds to meson Date: Wed, 17 May 2023 19:45:03 +0200 Message-Id: <20230517174520.887405-52-pbonzini@redhat.com> X-Mailer: git-send-email 2.40.1 In-Reply-To: <20230517174520.887405-1-pbonzini@redhat.com> References: <20230517174520.887405-1-pbonzini@redhat.com> MIME-Version: 1.0 Received-SPF: pass client-ip=170.10.129.124; envelope-from=pbonzini@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org QEMU adds the path to glib.h to all compilation commands. This is simpler due to the pervasive use of static_library, and was grandfathered in from the previous Make-based build system. Until Meson 0.63 the only way to do this was to detect glib in configure and use add_project_arguments, but now it is possible to use add_project_dependencies instead. gmodule is detected in a separate variable, with export enabled for modules and disabled for plugin. Reviewed-by: Philippe Mathieu-DaudĂ© Signed-off-by: Paolo Bonzini --- configure | 98 +--------------------------------------- contrib/plugins/Makefile | 4 +- meson.build | 92 +++++++++++++++++++++++++++++-------- util/meson.build | 4 +- 4 files changed, 80 insertions(+), 118 deletions(-) diff --git a/configure b/configure index 651ada7b6bc3..9b296c0ec132 100755 --- a/configure +++ b/configure @@ -231,7 +231,6 @@ safe_stack="" use_containers="yes" gdb_bin=$(command -v "gdb-multiarch" || command -v "gdb") gdb_arches="" -glib_has_gslice="no" if test -e "$source_path/.git" then @@ -1509,95 +1508,6 @@ if ! has "$pkg_config_exe"; then error_exit "pkg-config binary '$pkg_config_exe' not found" fi -########################################## -# glib support probe - -# When bumping glib_req_ver, please check also whether we should increase -# the _WIN32_WINNT setting in osdep.h according to the value from glib -glib_req_ver=2.56 -glib_modules=gthread-2.0 -if test "$modules" = yes; then - glib_modules="$glib_modules gmodule-export-2.0" -elif test "$plugins" = "yes"; then - glib_modules="$glib_modules gmodule-no-export-2.0" -fi - -for i in $glib_modules; do - if $pkg_config --atleast-version=$glib_req_ver $i; then - glib_cflags=$($pkg_config --cflags $i) - glib_libs=$($pkg_config --libs $i) - else - error_exit "glib-$glib_req_ver $i is required to compile QEMU" - fi -done - -# Check whether glib has gslice, which we have to avoid for correctness. -# TODO: remove this check and the corresponding workaround (qtree) when -# the minimum supported glib is >= $glib_dropped_gslice_version. -glib_dropped_gslice_version=2.75.3 -for i in $glib_modules; do - if ! $pkg_config --atleast-version=$glib_dropped_gslice_version $i; then - glib_has_gslice="yes" - break - fi -done - -glib_bindir="$($pkg_config --variable=bindir glib-2.0)" -if test -z "$glib_bindir" ; then - glib_bindir="$($pkg_config --variable=prefix glib-2.0)"/bin -fi - -# This workaround is required due to a bug in pkg-config file for glib as it -# doesn't define GLIB_STATIC_COMPILATION for pkg-config --static - -if test "$static" = yes && test "$mingw32" = yes; then - glib_cflags="-DGLIB_STATIC_COMPILATION $glib_cflags" -fi - -# Sanity check that the current size_t matches the -# size that glib thinks it should be. This catches -# problems on multi-arch where people try to build -# 32-bit QEMU while pointing at 64-bit glib headers -cat > $TMPC < -#include - -#define QEMU_BUILD_BUG_ON(x) \ - typedef char qemu_build_bug_on[(x)?-1:1] __attribute__((unused)); - -int main(void) { - QEMU_BUILD_BUG_ON(sizeof(size_t) != GLIB_SIZEOF_SIZE_T); - return 0; -} -EOF - -if ! compile_prog "$glib_cflags" "$glib_libs" ; then - error_exit "sizeof(size_t) doesn't match GLIB_SIZEOF_SIZE_T."\ - "You probably need to set PKG_CONFIG_LIBDIR"\ - "to point to the right pkg-config files for your"\ - "build target" -fi - -# Silence clang warnings triggered by glib < 2.57.2 -cat > $TMPC << EOF -#include -typedef struct Foo { - int i; -} Foo; -static void foo_free(Foo *f) -{ - g_free(f); -} -G_DEFINE_AUTOPTR_CLEANUP_FUNC(Foo, foo_free) -int main(void) { return 0; } -EOF -if ! compile_prog "$glib_cflags -Werror" "$glib_libs" ; then - if cc_has_warning_flag "-Wno-unused-function"; then - glib_cflags="$glib_cflags -Wno-unused-function" - CONFIGURE_CFLAGS="$CONFIGURE_CFLAGS -Wno-unused-function" - fi -fi - ########################################## # fdt probe @@ -2454,16 +2364,10 @@ echo "PYTHON=$python" >> $config_host_mak echo "GENISOIMAGE=$genisoimage" >> $config_host_mak echo "MESON=$meson" >> $config_host_mak echo "NINJA=$ninja" >> $config_host_mak +echo "PKG_CONFIG=${pkg_config_exe}" >> $config_host_mak echo "CC=$cc" >> $config_host_mak echo "QEMU_CFLAGS=$QEMU_CFLAGS" >> $config_host_mak echo "QEMU_OBJCFLAGS=$QEMU_OBJCFLAGS" >> $config_host_mak -echo "GLIB_CFLAGS=$glib_cflags" >> $config_host_mak -echo "GLIB_LIBS=$glib_libs" >> $config_host_mak -echo "GLIB_BINDIR=$glib_bindir" >> $config_host_mak -echo "GLIB_VERSION=$($pkg_config --modversion glib-2.0)" >> $config_host_mak -if test "$glib_has_gslice" = "yes" ; then - echo "HAVE_GLIB_WITH_SLICE_ALLOCATOR=y" >> $config_host_mak -fi echo "QEMU_LDFLAGS=$QEMU_LDFLAGS" >> $config_host_mak echo "EXESUF=$EXESUF" >> $config_host_mak diff --git a/contrib/plugins/Makefile b/contrib/plugins/Makefile index 23e0396687e8..8a316cd76f2f 100644 --- a/contrib/plugins/Makefile +++ b/contrib/plugins/Makefile @@ -3,7 +3,7 @@ # This Makefile example is fairly independent from the main makefile # so users can take and adapt it for their build. We only really # include config-host.mak so we don't have to repeat probing for -# cflags that the main configure has already done for us. +# programs that the main configure has already done for us. # BUILD_DIR := $(CURDIR)/../.. @@ -26,7 +26,7 @@ SONAMES := $(addsuffix .so,$(addprefix lib,$(NAMES))) # The main QEMU uses Glib extensively so it's perfectly fine to use it # in plugins (which many example do). -CFLAGS = $(GLIB_CFLAGS) +CFLAGS := $(shell $(PKG_CONFIG) --cflags glib-2.0) CFLAGS += -fPIC -Wall $(filter -W%, $(QEMU_CFLAGS)) CFLAGS += $(if $(findstring no-psabi,$(QEMU_CFLAGS)),-Wpsabi) CFLAGS += $(if $(CONFIG_DEBUG_TCG), -ggdb -O0) diff --git a/meson.build b/meson.build index 88fba57cca81..50e36729eb67 100644 --- a/meson.build +++ b/meson.build @@ -492,23 +492,79 @@ endif # Dependencies # ################ -# The path to glib.h is added to all compilation commands. This was -# grandfathered in from the QEMU Makefiles. -add_project_arguments(config_host['GLIB_CFLAGS'].split(), - native: false, language: all_languages) -glib = declare_dependency(compile_args: config_host['GLIB_CFLAGS'].split(), - link_args: config_host['GLIB_LIBS'].split(), - version: config_host['GLIB_VERSION'], - variables: { - 'bindir': config_host['GLIB_BINDIR'], - }) -# override glib dep with the configure results (for subprojects) -meson.override_dependency('glib-2.0', glib) -# pass down whether Glib has the slice allocator -if config_host.has_key('HAVE_GLIB_WITH_SLICE_ALLOCATOR') - config_host_data.set('HAVE_GLIB_WITH_SLICE_ALLOCATOR', true) +# When bumping glib minimum version, please check also whether to increase +# the _WIN32_WINNT setting in osdep.h according to the value from glib +glib_req_ver = '>=2.56.0' +glib_pc = dependency('glib-2.0', version: glib_req_ver, required: true, + method: 'pkg-config') +glib_cflags = [] +if config_host.has_key('CONFIG_MODULES') + gmodule = dependency('gmodule-export-2.0', version: glib_req_ver, required: true, + method: 'pkg-config') +elif config_host.has_key('CONFIG_PLUGIN') + gmodule = dependency('gmodule-no-export-2.0', version: glib_req_ver, required: true, + method: 'pkg-config') +else + gmodule = not_found endif +# This workaround is required due to a bug in pkg-config file for glib as it +# doesn't define GLIB_STATIC_COMPILATION for pkg-config --static +if targetos == 'windows' and get_option('prefer_static') + glib_cflags += ['-DGLIB_STATIC_COMPILATION'] +endif + +# Sanity check that the current size_t matches the +# size that glib thinks it should be. This catches +# problems on multi-arch where people try to build +# 32-bit QEMU while pointing at 64-bit glib headers + +if not cc.compiles(''' + #include + #include + + #define QEMU_BUILD_BUG_ON(x) \ + typedef char qemu_build_bug_on[(x)?-1:1] __attribute__((unused)); + + int main(void) { + QEMU_BUILD_BUG_ON(sizeof(size_t) != GLIB_SIZEOF_SIZE_T); + return 0; + }''', dependencies: glib_pc, args: glib_cflags) + error('''sizeof(size_t) doesn't match GLIB_SIZEOF_SIZE_T. + You probably need to set PKG_CONFIG_LIBDIR" to point + to the right pkg-config files for your build target.''') +endif + +# Silence clang warnings triggered by glib < 2.57.2 +if not cc.compiles(''' + #include + typedef struct Foo { + int i; + } Foo; + static void foo_free(Foo *f) + { + g_free(f); + } + G_DEFINE_AUTOPTR_CLEANUP_FUNC(Foo, foo_free) + int main(void) { return 0; }''', dependencies: glib_pc, args: ['-Werror']) + glib_cflags += cc.get_supported_arguments('-Wno-unused-function') +endif +glib = declare_dependency(dependencies: [glib_pc, gmodule], + compile_args: glib_cflags, + version: glib_pc.version()) + +# Check whether glib has gslice, which we have to avoid for correctness. +# TODO: remove this check and the corresponding workaround (qtree) when +# the minimum supported glib is >= 2.75.3 +glib_has_gslice = glib.version().version_compare('<2.75.3') + +# override glib dep to include the above refinements +meson.override_dependency('glib-2.0', glib) + +# The path to glib.h is added to all compilation commands. +add_project_dependencies(glib.partial_dependency(compile_args: true, includes: true), + native: false, language: all_languages) + gio = not_found gdbus_codegen = not_found gdbus_codegen_error = '@0@ requires gdbus-codegen, please install libgio' @@ -931,7 +987,7 @@ if have_system and get_option('curses').allowed() int main(void) { iconv_t conv = iconv_open("WCHAR_T", "UCS-2"); return conv != (iconv_t) -1; - }''', args: config_host['GLIB_CFLAGS'].split() + config_host['GLIB_LIBS'].split() + link_args) + }''', args: link_args, dependencies: glib) iconv = declare_dependency(link_args: link_args, dependencies: glib) break endif @@ -1962,6 +2018,7 @@ config_host_data.set('CONFIG_SYNC_FILE_RANGE', cc.has_function('sync_file_range' config_host_data.set('CONFIG_TIMERFD', cc.has_function('timerfd_create')) config_host_data.set('HAVE_COPY_FILE_RANGE', cc.has_function('copy_file_range')) config_host_data.set('HAVE_GETIFADDRS', cc.has_function('getifaddrs')) +config_host_data.set('HAVE_GLIB_WITH_SLICE_ALLOCATOR', glib_has_gslice) config_host_data.set('HAVE_OPENPTY', cc.has_function('openpty', dependencies: util)) config_host_data.set('HAVE_STRCHRNUL', cc.has_function('strchrnul')) config_host_data.set('HAVE_SYSTEM_FUNCTION', cc.has_function('system', prefix: '#include ')) @@ -3034,7 +3091,6 @@ subdir('ui') subdir('hw') subdir('gdbstub') - if enable_modules libmodulecommon = static_library('module-common', files('module-common.c') + genh, pic: true, c_args: '-DBUILD_DSO') modulecommon = declare_dependency(link_whole: libmodulecommon, compile_args: '-DBUILD_DSO') @@ -3656,7 +3712,7 @@ if host_machine.system() == 'windows' '@OUTPUT@', get_option('prefix'), meson.current_source_dir(), - config_host['GLIB_BINDIR'], + glib_pc.get_variable('bindir'), host_machine.cpu(), '--', '-DDISPLAYVERSION=' + meson.project_version(), diff --git a/util/meson.build b/util/meson.build index 3c2cfc6ede66..2cb103fc727e 100644 --- a/util/meson.build +++ b/util/meson.build @@ -26,7 +26,9 @@ util_ss.add(when: 'CONFIG_WIN32', if_true: files('oslib-win32.c')) util_ss.add(when: 'CONFIG_WIN32', if_true: files('qemu-thread-win32.c')) util_ss.add(when: 'CONFIG_WIN32', if_true: winmm) util_ss.add(when: 'CONFIG_WIN32', if_true: pathcch) -util_ss.add(when: 'HAVE_GLIB_WITH_SLICE_ALLOCATOR', if_true: files('qtree.c')) +if glib_has_gslice + util_ss.add(files('qtree.c')) +endif util_ss.add(files('envlist.c', 'path.c', 'module.c')) util_ss.add(files('host-utils.c')) util_ss.add(files('bitmap.c', 'bitops.c')) From patchwork Wed May 17 17:45:04 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Paolo Bonzini X-Patchwork-Id: 1782859 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=nongnu.org (client-ip=209.51.188.17; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=) Authentication-Results: legolas.ozlabs.org; dkim=pass (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=aH8R5jLq; dkim-atps=neutral Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-ECDSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4QM13W2gxkz20dX for ; Thu, 18 May 2023 03:55:19 +1000 (AEST) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1pzLFr-0003EZ-O4; Wed, 17 May 2023 13:48:03 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pzLEk-0001qT-Mz for qemu-devel@nongnu.org; Wed, 17 May 2023 13:47:00 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.129.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pzLEh-0004Ql-T8 for qemu-devel@nongnu.org; Wed, 17 May 2023 13:46:54 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1684345610; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=7FdptHKzmhUoRSDNT+ltsR8KpQAZl9TwWtsZXexWKtU=; b=aH8R5jLqKncsxtXgpGWFy6ZAxmNobCghvLVOwTMaes++ZI4EqHbEwNkNwEXpLEFtD0Rfnz RsWl6xvvAWHGHkToJuBgdk0Jo6KeP+0tSenu4jtN5AYY5VnKFuLK2nqmptecBLzLM77jRx bQvo4iVZDxBQQNGckugB1teSl79Mco0= Received: from mail-wr1-f71.google.com (mail-wr1-f71.google.com [209.85.221.71]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-461-qEIzbjZhOX2lp88THfOdzw-1; Wed, 17 May 2023 13:46:49 -0400 X-MC-Unique: qEIzbjZhOX2lp88THfOdzw-1 Received: by mail-wr1-f71.google.com with SMTP id ffacd0b85a97d-30479b764f9so451615f8f.0 for ; Wed, 17 May 2023 10:46:48 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1684345607; x=1686937607; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=7FdptHKzmhUoRSDNT+ltsR8KpQAZl9TwWtsZXexWKtU=; b=fZS4u/xrjxE8+b7dmTFHx5LXVsz82GgaZ+OBQ91+v692frqU0Lv+akIBMw+3zfP+mE iOTe2TOneKoyrfdLO04deRyE2v6fNe+NPdPKKFTNZZWRllbiT7th5Tn6IxoM+SZ6/3WA exa7GLKfb2Kqo8cKlFpSVPiuLDUvjnZWd50sq5P6L3qwn4gRvSCDCKh8p33sAETKrGww w+SYnDkUzx0AFNBn8bndgRpXOb7SgGYj+Atcjcf1+bM7rInQb+Xd942jvaxgFfqm9rO+ 7YMbGXLyPCj8K+1SyCLBHgai+i/AzE/9bGleggeKfy/lmxVJIrUZXQhUOS6ak9Gn7TmE 6cAw== X-Gm-Message-State: AC+VfDy5aOw08WiWqlmfB9x2ezCyP0lgaYXhtjXN215aDi/BxGLs6SZI qqCNfB6ERXySD08uiCJpgH/ZODrSEc+3KCIACTt4+z60lvm62DBLugfNhcsaGJOB8S1Xq60QPdn V9zohH3yl3kho6euqBzZP5/lBiC22EUD1Fy7vpwSQXoRLjpErLUlaz0Ty2LZiu63tXquj4jgDiL Q= X-Received: by 2002:a5d:4f84:0:b0:306:4063:1aff with SMTP id d4-20020a5d4f84000000b0030640631affmr1234127wru.30.1684345607448; Wed, 17 May 2023 10:46:47 -0700 (PDT) X-Google-Smtp-Source: ACHHUZ733J1oxDFfss54C65ITeNkvnDo2v3D95VZ5LS4Po4EvtWEPpAUFSWewV4jDivbxUI1Bb7g1A== X-Received: by 2002:a5d:4f84:0:b0:306:4063:1aff with SMTP id d4-20020a5d4f84000000b0030640631affmr1234112wru.30.1684345607037; Wed, 17 May 2023 10:46:47 -0700 (PDT) Received: from [192.168.10.118] ([2001:b07:6468:f312:9af8:e5f5:7516:fa89]) by smtp.gmail.com with ESMTPSA id u12-20020a05600c00cc00b003f4ecf1fcbcsm2911460wmm.22.2023.05.17.10.46.46 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 17 May 2023 10:46:46 -0700 (PDT) From: Paolo Bonzini To: qemu-devel@nongnu.org Cc: =?utf-8?q?Marc-Andr=C3=A9_Lureau?= Subject: [PULL 52/68] configure: remove pkg-config functions Date: Wed, 17 May 2023 19:45:04 +0200 Message-Id: <20230517174520.887405-53-pbonzini@redhat.com> X-Mailer: git-send-email 2.40.1 In-Reply-To: <20230517174520.887405-1-pbonzini@redhat.com> References: <20230517174520.887405-1-pbonzini@redhat.com> MIME-Version: 1.0 Received-SPF: pass client-ip=170.10.129.124; envelope-from=pbonzini@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org All uses of pkg-config have been moved to Meson. Reviewed-by: Marc-AndrĂ© Lureau Signed-off-by: Paolo Bonzini --- configure | 21 ++++----------------- docs/devel/build-system.rst | 4 ---- 2 files changed, 4 insertions(+), 21 deletions(-) diff --git a/configure b/configure index 9b296c0ec132..3267b1027970 100755 --- a/configure +++ b/configure @@ -389,11 +389,7 @@ strip="${STRIP-${cross_prefix}strip}" widl="${WIDL-${cross_prefix}widl}" windres="${WINDRES-${cross_prefix}windres}" windmc="${WINDMC-${cross_prefix}windmc}" -pkg_config_exe="${PKG_CONFIG-${cross_prefix}pkg-config}" -query_pkg_config() { - "${pkg_config_exe}" ${QEMU_PKG_CONFIG_FLAGS} "$@" -} -pkg_config=query_pkg_config +pkg_config="${PKG_CONFIG-${cross_prefix}pkg-config}" sdl2_config="${SDL2_CONFIG-${cross_prefix}sdl2-config}" # default flags for all hosts @@ -791,9 +787,7 @@ for opt do ;; --without-default-features) # processed above ;; - --static) - static="yes" - QEMU_PKG_CONFIG_FLAGS="--static $QEMU_PKG_CONFIG_FLAGS" + --static) static="yes" ;; --bindir=*) bindir="$optarg" ;; @@ -1501,13 +1495,6 @@ EOF fi fi -########################################## -# pkg-config probe - -if ! has "$pkg_config_exe"; then - error_exit "pkg-config binary '$pkg_config_exe' not found" -fi - ########################################## # fdt probe @@ -2364,7 +2351,7 @@ echo "PYTHON=$python" >> $config_host_mak echo "GENISOIMAGE=$genisoimage" >> $config_host_mak echo "MESON=$meson" >> $config_host_mak echo "NINJA=$ninja" >> $config_host_mak -echo "PKG_CONFIG=${pkg_config_exe}" >> $config_host_mak +echo "PKG_CONFIG=${pkg_config}" >> $config_host_mak echo "CC=$cc" >> $config_host_mak echo "QEMU_CFLAGS=$QEMU_CFLAGS" >> $config_host_mak echo "QEMU_OBJCFLAGS=$QEMU_OBJCFLAGS" >> $config_host_mak @@ -2514,7 +2501,7 @@ if test "$skip_meson" = no; then test -n "$objcc" && echo "objc = [$(meson_quote $objcc $CPU_CFLAGS)]" >> $cross echo "ar = [$(meson_quote $ar)]" >> $cross echo "nm = [$(meson_quote $nm)]" >> $cross - echo "pkgconfig = [$(meson_quote $pkg_config_exe)]" >> $cross + echo "pkgconfig = [$(meson_quote $pkg_config)]" >> $cross echo "ranlib = [$(meson_quote $ranlib)]" >> $cross if has $sdl2_config; then echo "sdl2-config = [$(meson_quote $sdl2_config)]" >> $cross diff --git a/docs/devel/build-system.rst b/docs/devel/build-system.rst index 9db18aff159e..66cfe7b8bdc8 100644 --- a/docs/devel/build-system.rst +++ b/docs/devel/build-system.rst @@ -103,10 +103,6 @@ developers in checking for system features: Print $MESSAGE to stderr, followed by $MORE... and then exit from the configure script with non-zero status -``query_pkg_config $ARGS...`` - Run pkg-config passing it $ARGS. If QEMU is doing a static build, - then --static will be automatically added to $ARGS - Stage 2: Meson ============== From patchwork Wed May 17 17:45:05 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Paolo Bonzini X-Patchwork-Id: 1782848 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=nongnu.org (client-ip=209.51.188.17; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=) Authentication-Results: legolas.ozlabs.org; dkim=pass (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=J8Q/Dodq; dkim-atps=neutral Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-ECDSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4QM0yw3FBtz20dq for ; Thu, 18 May 2023 03:51:20 +1000 (AEST) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1pzLGx-0004xb-4C; Wed, 17 May 2023 13:49:11 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pzLEm-0001qX-Do for qemu-devel@nongnu.org; Wed, 17 May 2023 13:47:00 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pzLEk-0004RJ-En for qemu-devel@nongnu.org; Wed, 17 May 2023 13:46:56 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1684345613; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=3Z22DnnWCfFDBaUJH5DlfNaapeEc3FSg7euNvn0EgDY=; b=J8Q/DodqT5NBFctgujHB7UaQJe0YsCTeUQ6c/4BhPNXoUW+GlamuXbBwIj/Jfoor9btOj0 lkOgSul3e+fNKRWxp9A7H9gCTdFiP1tYxsCwnYBx3Ny+fPMRSiQF3eAY0btHkBfEDN1JKc ylEXHsXHNtD/xZwGWv0mjFAJxDHExng= Received: from mail-wr1-f71.google.com (mail-wr1-f71.google.com [209.85.221.71]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-170-tTOj0D6UO3GXJvYigjChSA-1; Wed, 17 May 2023 13:46:51 -0400 X-MC-Unique: tTOj0D6UO3GXJvYigjChSA-1 Received: by mail-wr1-f71.google.com with SMTP id ffacd0b85a97d-3079d15b51dso166092f8f.1 for ; Wed, 17 May 2023 10:46:51 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1684345609; x=1686937609; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=3Z22DnnWCfFDBaUJH5DlfNaapeEc3FSg7euNvn0EgDY=; b=Hx3XfXncFQRgeOyuK3CLqmgwlRTm6CWOTNhVk7lgkIN2YMc0vef9DLHfqUALis2MC1 5HHpR88aVs7/w19B2aVPUCOMXel0tT/uwvAUvuOVQ6KqVvIABD7dJSqGzmMJQICNtia2 hkOgFWJG4xzKyI6bnklblJOvqMWC7SyFw5yrSvhzmpxT9a8lft3roY6/zS1iCsu1cpGr /KuYPQmktz+o7Qn4EB/FI+v3rPn5qn3N2IDrxn7Y9DPX7NZvVa88FQUI+isjOlUnlaRe IN3URieT65sElTgQSQZrZtuaQsc+vtOhEG7QLcFaYTxs001RJgM/OrY5Bi+Z77mjRniG VnHg== X-Gm-Message-State: AC+VfDxPRg1S2llS1eSCehibf8DHqil08bSgTIm7//wW13ElmlJ3ec9y 3hpu3ZCl/niqT7oX9f/TM25chbjdYdShyWcJRH4MDvHmeI15RQvx75UeUgisVg3e77yoHd8+wDd /IxtpQ70N6mCwRbR6WMJyt5chfgDj8qh/Y48SM9MJ55AVpAabM8uASUmZU0bwAvN1OTJ8nybdJD w= X-Received: by 2002:a5d:4212:0:b0:306:45ef:9935 with SMTP id n18-20020a5d4212000000b0030645ef9935mr1479148wrq.13.1684345609444; Wed, 17 May 2023 10:46:49 -0700 (PDT) X-Google-Smtp-Source: ACHHUZ4mA961CiSG6DeCgBMYWe8+ggHZOY2YcoNrntrlh9iP8yTI1l27hGORroY0dMGIbsNdJwMqFA== X-Received: by 2002:a5d:4212:0:b0:306:45ef:9935 with SMTP id n18-20020a5d4212000000b0030645ef9935mr1479136wrq.13.1684345609027; Wed, 17 May 2023 10:46:49 -0700 (PDT) Received: from [192.168.10.118] ([2001:b07:6468:f312:9af8:e5f5:7516:fa89]) by smtp.gmail.com with ESMTPSA id s2-20020a1cf202000000b003f50876905dsm2876986wmc.6.2023.05.17.10.46.47 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 17 May 2023 10:46:48 -0700 (PDT) From: Paolo Bonzini To: qemu-devel@nongnu.org Cc: =?utf-8?q?Marc-Andr=C3=A9_Lureau?= Subject: [PULL 53/68] configure, meson: move --enable-modules to Meson Date: Wed, 17 May 2023 19:45:05 +0200 Message-Id: <20230517174520.887405-54-pbonzini@redhat.com> X-Mailer: git-send-email 2.40.1 In-Reply-To: <20230517174520.887405-1-pbonzini@redhat.com> References: <20230517174520.887405-1-pbonzini@redhat.com> MIME-Version: 1.0 Received-SPF: pass client-ip=170.10.133.124; envelope-from=pbonzini@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Reviewed-by: Marc-AndrĂ© Lureau Signed-off-by: Paolo Bonzini --- configure | 20 -------------------- meson.build | 22 ++++++++++++++-------- meson_options.txt | 2 ++ scripts/meson-buildoptions.sh | 3 +++ tests/qtest/meson.build | 2 +- 5 files changed, 20 insertions(+), 29 deletions(-) diff --git a/configure b/configure index 3267b1027970..5edd3765cd4e 100755 --- a/configure +++ b/configure @@ -280,7 +280,6 @@ tsan="no" fortify_source="yes" docs="auto" EXESUF="" -modules="no" prefix="/usr/local" qemu_suffix="qemu" softmmu="yes" @@ -751,12 +750,6 @@ for opt do ;; --disable-docs) docs=disabled ;; - --enable-modules) - modules="yes" - ;; - --disable-modules) - modules="no" - ;; --cpu=*) ;; --target-list=*) target_list="$optarg" @@ -1051,7 +1044,6 @@ cat << EOF linux-user all linux usermode emulation targets bsd-user all BSD usermode emulation targets pie Position Independent Executables - modules modules support (non-Windows) debug-tcg TCG debugging (default is disabled) debug-info debugging information safe-stack SafeStack Stack Smash Protection. Depends on @@ -1342,16 +1334,7 @@ EOF fi fi -# Our module code doesn't support Windows -if test "$modules" = "yes" && test "$mingw32" = "yes" ; then - error_exit "Modules are not available for Windows" -fi - -# Static linking is not possible with plugins, modules or PIE if test "$static" = "yes" ; then - if test "$modules" = "yes" ; then - error_exit "static and modules are mutually incompatible" - fi if test "$plugins" = "yes"; then error_exit "static and plugins are mutually incompatible" else @@ -2308,9 +2291,6 @@ if test "$solaris" = "yes" ; then fi echo "SRC_PATH=$source_path" >> $config_host_mak echo "TARGET_DIRS=$target_list" >> $config_host_mak -if test "$modules" = "yes"; then - echo "CONFIG_MODULES=y" >> $config_host_mak -fi # XXX: suppress that if [ "$bsd" = "yes" ] ; then diff --git a/meson.build b/meson.build index 50e36729eb67..531f7a84c9aa 100644 --- a/meson.build +++ b/meson.build @@ -17,7 +17,6 @@ fs = import('fs') targetos = host_machine.system() sh = find_program('sh') config_host = keyval.load(meson.current_build_dir() / 'config-host.mak') -enable_modules = 'CONFIG_MODULES' in config_host cc = meson.get_compiler('c') all_languages = ['c'] @@ -84,6 +83,12 @@ have_ga = get_option('guest_agent') \ .require(targetos in ['sunos', 'linux', 'windows', 'freebsd', 'netbsd', 'openbsd'], error_message: 'unsupported OS for QEMU guest agent') \ .allowed() +enable_modules = get_option('modules') \ + .require(targetos != 'windows', + error_message: 'Modules are not available for Windows') \ + .require(not get_option('prefer_static'), + error_message: 'Modules are incompatible with static linking') \ + .allowed() have_block = have_system or have_tools python = import('python').find_installation() @@ -498,7 +503,7 @@ glib_req_ver = '>=2.56.0' glib_pc = dependency('glib-2.0', version: glib_req_ver, required: true, method: 'pkg-config') glib_cflags = [] -if config_host.has_key('CONFIG_MODULES') +if enable_modules gmodule = dependency('gmodule-export-2.0', version: glib_req_ver, required: true, method: 'pkg-config') elif config_host.has_key('CONFIG_PLUGIN') @@ -1725,7 +1730,7 @@ if get_option('cfi') if not get_option('b_lto') error('Selected Control-Flow Integrity but LTO is disabled') endif - if config_host.has_key('CONFIG_MODULES') + if enable_modules error('Selected Control-Flow Integrity is not compatible with modules') endif # Check for cfi flags. CFI requires LTO so we can't use @@ -1823,7 +1828,7 @@ config_host_data.set_quoted('CONFIG_QEMU_LOCALSTATEDIR', get_option('prefix') / config_host_data.set_quoted('CONFIG_QEMU_MODDIR', get_option('prefix') / qemu_moddir) config_host_data.set_quoted('CONFIG_SYSCONFDIR', get_option('prefix') / get_option('sysconfdir')) -if config_host.has_key('CONFIG_MODULES') +if enable_modules config_host_data.set('CONFIG_STAMP', run_command( meson.current_source_dir() / 'scripts/qemu-stamp.py', meson.project_version(), get_option('pkgversion'), '--', @@ -1886,6 +1891,7 @@ config_host_data.set('CONFIG_LIBSSH', libssh.found()) config_host_data.set('CONFIG_LINUX_AIO', libaio.found()) config_host_data.set('CONFIG_LINUX_IO_URING', linux_io_uring.found()) config_host_data.set('CONFIG_LIBPMEM', libpmem.found()) +config_host_data.set('CONFIG_MODULES', enable_modules) config_host_data.set('CONFIG_NUMA', numa.found()) if numa.found() config_host_data.set('HAVE_NUMA_HAS_PREFERRED_MANY', @@ -3233,7 +3239,7 @@ foreach d, list : modules endif foreach m, module_ss : list - if enable_modules and targetos != 'windows' + if enable_modules module_ss = module_ss.apply(config_all, strict: false) sl = static_library(d + '-' + m, [genh, module_ss.sources()], dependencies: [modulecommon, module_ss.dependencies()], pic: true) @@ -3266,7 +3272,7 @@ endforeach foreach d, list : target_modules foreach m, module_ss : list - if enable_modules and targetos != 'windows' + if enable_modules foreach target : target_dirs if target.endswith('-softmmu') config_target = config_target_mak[target] @@ -3786,8 +3792,8 @@ summary_info += {'system-mode emulation': have_system} summary_info += {'user-mode emulation': have_user} summary_info += {'block layer': have_block} summary_info += {'Install blobs': get_option('install_blobs')} -summary_info += {'module support': config_host.has_key('CONFIG_MODULES')} -if config_host.has_key('CONFIG_MODULES') +summary_info += {'module support': enable_modules} +if enable_modules summary_info += {'alternative module path': get_option('module_upgrades')} endif summary_info += {'fuzzing support': get_option('fuzzing')} diff --git a/meson_options.txt b/meson_options.txt index a350520f6abf..a1240e0b2a59 100644 --- a/meson_options.txt +++ b/meson_options.txt @@ -42,6 +42,8 @@ option('fuzzing', type : 'boolean', value: false, description: 'build fuzzing targets') option('gettext', type : 'feature', value : 'auto', description: 'Localization of the GTK+ user interface') +option('modules', type : 'feature', value : 'disabled', + description: 'modules support (non Windows)') option('module_upgrades', type : 'boolean', value : false, description: 'try to load modules from alternate paths for upgrades') option('install_blobs', type : 'boolean', value : true, diff --git a/scripts/meson-buildoptions.sh b/scripts/meson-buildoptions.sh index 10c4969a013e..43301864e395 100644 --- a/scripts/meson-buildoptions.sh +++ b/scripts/meson-buildoptions.sh @@ -127,6 +127,7 @@ meson_options_help() { printf "%s\n" ' lzo lzo compression support' printf "%s\n" ' malloc-trim enable libc malloc_trim() for memory optimization' printf "%s\n" ' membarrier membarrier system call (for Linux 4.14+ or Windows' + printf "%s\n" ' modules modules support (non Windows)' printf "%s\n" ' mpath Multipath persistent reservation passthrough' printf "%s\n" ' multiprocess Out of process device emulation support' printf "%s\n" ' netmap netmap network backend support' @@ -357,6 +358,8 @@ _meson_option_parse() { --disable-membarrier) printf "%s" -Dmembarrier=disabled ;; --enable-module-upgrades) printf "%s" -Dmodule_upgrades=true ;; --disable-module-upgrades) printf "%s" -Dmodule_upgrades=false ;; + --enable-modules) printf "%s" -Dmodules=enabled ;; + --disable-modules) printf "%s" -Dmodules=disabled ;; --enable-mpath) printf "%s" -Dmpath=enabled ;; --disable-mpath) printf "%s" -Dmpath=disabled ;; --enable-multiprocess) printf "%s" -Dmultiprocess=enabled ;; diff --git a/tests/qtest/meson.build b/tests/qtest/meson.build index 48cd35b5b203..ab422772d310 100644 --- a/tests/qtest/meson.build +++ b/tests/qtest/meson.build @@ -23,7 +23,7 @@ qtests_generic = [ 'readconfig-test', 'netdev-socket', ] -if config_host.has_key('CONFIG_MODULES') +if enable_modules qtests_generic += [ 'modules-test' ] endif From patchwork Wed May 17 17:45:06 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Paolo Bonzini X-Patchwork-Id: 1782852 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=nongnu.org (client-ip=209.51.188.17; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=) Authentication-Results: legolas.ozlabs.org; dkim=pass (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=bxkSlmOO; dkim-atps=neutral Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-ECDSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4QM10H2WHBz20KF for ; Thu, 18 May 2023 03:52:31 +1000 (AEST) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1pzLGm-0004Cs-Np; Wed, 17 May 2023 13:49:03 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pzLEn-0001sQ-Au for qemu-devel@nongnu.org; Wed, 17 May 2023 13:47:02 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.129.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pzLEk-0004RV-UK for qemu-devel@nongnu.org; Wed, 17 May 2023 13:46:57 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1684345614; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=uMLDcjqwDF35UrdRcpPmiRXSC/LtqcCJfvAbqBE3Sw4=; b=bxkSlmOOEoVPJvjv0lgGggcuaBp/pa3KgNh8o0sZDo14vC6oPOn+0yOIzEvybjjYCkZZyJ 8Oud1TfyXI1dwD5PDDvfzpsPD2v48VPgPzXpdAWYE1L6RWfQ9lDrvsGvMsDsxUtpOqccpm biV/vtiD+Kg/PqLB+mFcHXnBR42sUbQ= Received: from mail-wm1-f70.google.com (mail-wm1-f70.google.com [209.85.128.70]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-595-8RLDpEesOCqGtX-dvC4cig-1; Wed, 17 May 2023 13:46:53 -0400 X-MC-Unique: 8RLDpEesOCqGtX-dvC4cig-1 Received: by mail-wm1-f70.google.com with SMTP id 5b1f17b1804b1-3f426d4944fso4324735e9.1 for ; Wed, 17 May 2023 10:46:53 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1684345611; x=1686937611; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=uMLDcjqwDF35UrdRcpPmiRXSC/LtqcCJfvAbqBE3Sw4=; b=EgHsaOylgNrHvFv42G9S1mWq+gWVNXBmGxyiz+eYlq0fc3mNK85jcXLnteIOZUN/R4 tTi1lxwErp5aDodeLK7SSUirw5FwcNJ0fhaz+PCKmWsOUrVh0MVsUvE4cpAPEd/L3nlL AJU/w9/Z2Uh5lgaqR0iS9rYVQeMaEdND1T8n6gqRBx1S3hERL3TBW9cQ2m9fiYLsJrOO 9j4ua+JGE4e0GxJkFQ7Zz+7GJuKUR3JIj7TRyvMSazny0S46hr6iv3VkYBYAvCdHINEM E50Q2Ibc4tzFEqpm6ummQ70Q25uGoGyDcqojuTZNp6JwbSBWYkMQ1Fslj2Aqy4zWCo9y SsNQ== X-Gm-Message-State: AC+VfDxd+I3Yv11EKl9ULKmwi6FhoIubpeE63E2cKroFuyWF6py7BnoP xJ+uT+vhEEiaC5RMPeWyNx83GPjNp1ARrXw/EKwT1ez9BswglgKb+atkIuLXQ3/M/88lhXRinnw Svml+pvwRw9xPb2ukyt89XlbV0DpoBWCKDjohi5++Gxm/QKl1p281eop3kwGR5Dxf7STO0KB8/c c= X-Received: by 2002:a1c:7507:0:b0:3f1:9acf:8682 with SMTP id o7-20020a1c7507000000b003f19acf8682mr26759930wmc.17.1684345611464; Wed, 17 May 2023 10:46:51 -0700 (PDT) X-Google-Smtp-Source: ACHHUZ7Oq4Z6v3OC/KDEy3PEPoX1r6ZlU3Pxr0O8QmMS8lqcv/RDfL79kREE55HRiaNB6L+l2avYlA== X-Received: by 2002:a1c:7507:0:b0:3f1:9acf:8682 with SMTP id o7-20020a1c7507000000b003f19acf8682mr26759921wmc.17.1684345611064; Wed, 17 May 2023 10:46:51 -0700 (PDT) Received: from [192.168.10.118] ([2001:b07:6468:f312:9af8:e5f5:7516:fa89]) by smtp.gmail.com with ESMTPSA id l1-20020a7bc341000000b003f31cb7a203sm3065845wmj.14.2023.05.17.10.46.49 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 17 May 2023 10:46:50 -0700 (PDT) From: Paolo Bonzini To: qemu-devel@nongnu.org Cc: =?utf-8?q?Marc-Andr=C3=A9_Lureau?= Subject: [PULL 54/68] meson: prepare move of QEMU_CFLAGS to meson Date: Wed, 17 May 2023 19:45:06 +0200 Message-Id: <20230517174520.887405-55-pbonzini@redhat.com> X-Mailer: git-send-email 2.40.1 In-Reply-To: <20230517174520.887405-1-pbonzini@redhat.com> References: <20230517174520.887405-1-pbonzini@redhat.com> MIME-Version: 1.0 Received-SPF: pass client-ip=170.10.129.124; envelope-from=pbonzini@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Clean up the handling of compiler flags in meson.build, splitting the general flags that should be included in subprojects as well, from warning flags that only apply to QEMU itself. The two were mixed in both configure tests and meson tests. This split makes it easier to move the compiler tests piecewise from configure to Meson. Reviewed-by: Marc-AndrĂ© Lureau Signed-off-by: Paolo Bonzini --- meson.build | 53 +++++++++++++++++++++++++++++------------------------ 1 file changed, 29 insertions(+), 24 deletions(-) diff --git a/meson.build b/meson.build index 531f7a84c9aa..f162ed4e335f 100644 --- a/meson.build +++ b/meson.build @@ -190,10 +190,23 @@ endif # Compiler flags # ################## -qemu_cflags = config_host['QEMU_CFLAGS'].split() +qemu_common_flags = [] +qemu_cflags = [] +foreach arg : config_host['QEMU_CFLAGS'].split() + if arg.startswith('-W') + qemu_cflags += arg + else + qemu_common_flags += arg + endif +endforeach qemu_objcflags = config_host['QEMU_OBJCFLAGS'].split() qemu_ldflags = config_host['QEMU_LDFLAGS'].split() +if get_option('gprof') + qemu_common_flags += ['-p'] + qemu_ldflags += ['-p'] +endif + if get_option('prefer_static') qemu_ldflags += get_option('b_pie') ? '-static-pie' : '-static' endif @@ -207,10 +220,9 @@ if targetos == 'windows' qemu_ldflags += cc.get_supported_link_arguments('-Wl,--dynamicbase', '-Wl,--high-entropy-va') endif -if get_option('gprof') - qemu_cflags += ['-p'] - qemu_objcflags += ['-p'] - qemu_ldflags += ['-p'] +# Exclude --warn-common with TSan to suppress warnings from the TSan libraries. +if targetos != 'sunos' and not config_host.has_key('CONFIG_TSAN') + qemu_ldflags += cc.get_supported_link_arguments('-Wl,--warn-common') endif # Specify linker-script with add_project_link_arguments so that it is not placed @@ -226,8 +238,7 @@ if get_option('fuzzing') name: '-fsanitize-coverage-allowlist=/dev/null', args: ['-fsanitize-coverage-allowlist=/dev/null', '-fsanitize-coverage=trace-pc'] ) - add_global_arguments('-fsanitize-coverage-allowlist=instrumentation-filter', - native: false, language: all_languages) + qemu_common_flags += ['-fsanitize-coverage-allowlist=instrumentation-filter'] endif if get_option('fuzzing_engine') == '' @@ -235,10 +246,8 @@ if get_option('fuzzing') # compiled code. To build non-fuzzer binaries with --enable-fuzzing, link # everything with fsanitize=fuzzer-no-link. Otherwise, the linker will be # unable to bind the fuzzer-related callbacks added by instrumentation. - add_global_arguments('-fsanitize=fuzzer-no-link', - native: false, language: all_languages) - add_global_link_arguments('-fsanitize=fuzzer-no-link', - native: false, language: all_languages) + qemu_common_flags += ['-fsanitize=fuzzer-no-link'] + qemu_ldflags += ['-fsanitize=fuzzer-no-link'] # For the actual fuzzer binaries, we need to link against the libfuzzer # library. They need to be configurable, to support OSS-Fuzz fuzz_exe_ldflags = ['-fsanitize=fuzzer'] @@ -249,6 +258,9 @@ if get_option('fuzzing') endif endif +add_global_arguments(qemu_common_flags, native: false, language: all_languages) +add_global_link_arguments(qemu_ldflags, native: false, language: all_languages) + # Check that the C++ compiler exists and works with the C compiler. link_language = 'c' linker = cc @@ -272,16 +284,9 @@ if 'cpp' in all_languages endif endif -# Exclude --warn-common with TSan to suppress warnings from the TSan libraries. -if targetos != 'sunos' and not config_host.has_key('CONFIG_TSAN') - qemu_ldflags += linker.get_supported_link_arguments('-Wl,--warn-common') -endif - -add_global_link_arguments(qemu_ldflags, native: false, language: all_languages) - -add_global_arguments(qemu_cflags, native: false, language: 'c') -add_global_arguments(qemu_cxxflags, native: false, language: 'cpp') -add_global_arguments(qemu_objcflags, native: false, language: 'objc') +add_project_arguments(qemu_cflags, native: false, language: 'c') +add_project_arguments(qemu_cxxflags, native: false, language: 'cpp') +add_project_arguments(qemu_objcflags, native: false, language: 'objc') if targetos == 'linux' add_project_arguments('-isystem', meson.current_source_dir() / 'linux-headers', '-isystem', 'linux-headers', @@ -3844,12 +3849,12 @@ link_args = get_option(link_language + '_link_args') if link_args.length() > 0 summary_info += {'LDFLAGS': ' '.join(link_args)} endif -summary_info += {'QEMU_CFLAGS': ' '.join(qemu_cflags)} +summary_info += {'QEMU_CFLAGS': ' '.join(qemu_common_flags + qemu_cflags)} if 'cpp' in all_languages - summary_info += {'QEMU_CXXFLAGS': ' '.join(qemu_cxxflags)} + summary_info += {'QEMU_CXXFLAGS': ' '.join(qemu_common_flags + qemu_cxxflags)} endif if 'objc' in all_languages - summary_info += {'QEMU_OBJCFLAGS': ' '.join(qemu_objcflags)} + summary_info += {'QEMU_OBJCFLAGS': ' '.join(qemu_common_flags + qemu_objcflags)} endif summary_info += {'QEMU_LDFLAGS': ' '.join(qemu_ldflags)} summary_info += {'profiler': get_option('profiler')} From patchwork Wed May 17 17:45:07 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Paolo Bonzini X-Patchwork-Id: 1782887 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=nongnu.org (client-ip=209.51.188.17; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=) Authentication-Results: legolas.ozlabs.org; dkim=pass (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=Fo/aQLFT; dkim-atps=neutral Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-ECDSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4QM1FJ5Dpsz20dr for ; Thu, 18 May 2023 04:03:48 +1000 (AEST) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1pzLH5-0005My-HH; Wed, 17 May 2023 13:49:19 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pzLEr-0001x4-1f for qemu-devel@nongnu.org; Wed, 17 May 2023 13:47:04 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.129.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pzLEn-0004S2-H9 for qemu-devel@nongnu.org; Wed, 17 May 2023 13:46:59 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1684345616; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=9n4Ed9dRUkK2DxQPd4mYkI+hgP4LpPaX90+8tZBLRiY=; b=Fo/aQLFTw8i65CoT4/ltxuLl5bKYvHxecZHJMcPy/C5P4T3EAec3Uic7N8EovfO5u703Ly xjEDwwSpWqaSB2BIvsEOhofCYmmtF8ZT2p96wFQht5GyGrsAVhvaOjN5XyWlxrXFc/j1Yc f0CkCBGFKk0PYCrdDQotRLfsC9i+DlM= Received: from mail-wm1-f71.google.com (mail-wm1-f71.google.com [209.85.128.71]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-569-gdmVVqa0OXqcWHI59Q0gFA-1; Wed, 17 May 2023 13:46:55 -0400 X-MC-Unique: gdmVVqa0OXqcWHI59Q0gFA-1 Received: by mail-wm1-f71.google.com with SMTP id 5b1f17b1804b1-3f423d47945so4494465e9.0 for ; Wed, 17 May 2023 10:46:54 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1684345613; x=1686937613; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=9n4Ed9dRUkK2DxQPd4mYkI+hgP4LpPaX90+8tZBLRiY=; b=DGs8BYsf4lEfwE6k9Zfef6w/eo8q1877IiAzx07ylHe0H5+C0wyLiMGnSsKeFQqf73 UjMb75T7yJwwkTajyHrPE9yMEInrSMSNRp2t/cZ8LRhhqz0WS5y6IOYlkDZQe7RVW54r P4onKqJvFcaUk80TPedb+DzC3edyN1ya763sdPzBZuXCRnOSA7M2a89GIk/eXrypwFUG PS1uSoDAyks5QbGJy21kocr2CGTbWfcCOUhOAMiIFjdUKv2gLnLlG1MM1UhIA1t0B2Z4 J8ZjrgNh6IduQvWoT3rFSWWaWRA1ad3UoiQrO5PTMWvZBT0OD/L603J82IA1grSMcQ8G 6YhQ== X-Gm-Message-State: AC+VfDyvgarzfxD6Vzv8qYurSBM0/8Y3A9DmBxdM8iwxD/K/YowERE05 p7eldi6DMYIVGlWCLoZQGKqKKH0oe/dFOHHFR5IYlO6WRJIx4oCTKDm1PfuuVZHaCv3B2GmdEVm JztMYhyQyuKZp/p7596u7GZHDjqxY4RW0MwKPkHSjj7SPI503aS3tDStJWvqIafbJcOykdETFfK Y= X-Received: by 2002:a05:600c:b4d:b0:3f4:253b:92a7 with SMTP id k13-20020a05600c0b4d00b003f4253b92a7mr23666818wmr.30.1684345613505; Wed, 17 May 2023 10:46:53 -0700 (PDT) X-Google-Smtp-Source: ACHHUZ7xiuBVvoajB1NDMqWYWcUVQuArM+vFtDo2g61w9ZKZspn1burIDeAyJfCLLQwks6jFrrHZGQ== X-Received: by 2002:a05:600c:b4d:b0:3f4:253b:92a7 with SMTP id k13-20020a05600c0b4d00b003f4253b92a7mr23666794wmr.30.1684345612862; Wed, 17 May 2023 10:46:52 -0700 (PDT) Received: from [192.168.10.118] ([2001:b07:6468:f312:9af8:e5f5:7516:fa89]) by smtp.gmail.com with ESMTPSA id e2-20020a5d4e82000000b00309257ad16csm3383628wru.29.2023.05.17.10.46.51 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 17 May 2023 10:46:52 -0700 (PDT) From: Paolo Bonzini To: qemu-devel@nongnu.org Subject: [PULL 55/68] build: move sanitizer tests to meson Date: Wed, 17 May 2023 19:45:07 +0200 Message-Id: <20230517174520.887405-56-pbonzini@redhat.com> X-Mailer: git-send-email 2.40.1 In-Reply-To: <20230517174520.887405-1-pbonzini@redhat.com> References: <20230517174520.887405-1-pbonzini@redhat.com> MIME-Version: 1.0 Received-SPF: pass client-ip=170.10.129.124; envelope-from=pbonzini@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Signed-off-by: Paolo Bonzini --- configure | 147 --------------------------------- docs/devel/build-system.rst | 4 - meson.build | 56 ++++++++++++- meson_options.txt | 4 + scripts/meson-buildoptions.sh | 6 ++ tests/qemu-iotests/meson.build | 2 +- tests/unit/meson.build | 2 +- 7 files changed, 66 insertions(+), 155 deletions(-) diff --git a/configure b/configure index 5edd3765cd4e..34b26c45cf39 100755 --- a/configure +++ b/configure @@ -275,9 +275,6 @@ EXTRA_OBJCFLAGS="" EXTRA_LDFLAGS="" debug_tcg="no" -sanitizers="no" -tsan="no" -fortify_source="yes" docs="auto" EXESUF="" prefix="/usr/local" @@ -416,14 +413,6 @@ EOF compile_object } -check_include() { -cat > $TMPC < -int main(void) { return 0; } -EOF - compile_object -} - write_c_skeleton() { cat > $TMPC < /dev/null 2>&1 ; then - fortify_source="no"; - elif test -n "$cxx" && has $cxx && - echo | $cxx -dM -E - | grep __clang__ >/dev/null 2>&1 ; then - fortify_source="no"; - else - fortify_source="yes" - fi -fi - -########################################## -# checks for sanitizers - -have_asan=no -have_ubsan=no -have_asan_iface_h=no -have_asan_iface_fiber=no - -if test "$sanitizers" = "yes" ; then - write_c_skeleton - if compile_prog "$CPU_CFLAGS -Werror -fsanitize=address" ""; then - have_asan=yes - fi - - # we could use a simple skeleton for flags checks, but this also - # detect the static linking issue of ubsan, see also: - # https://gcc.gnu.org/bugzilla/show_bug.cgi?id=84285 - cat > $TMPC << EOF -int main(int argc, char **argv) -{ - return argc + 1; -} -EOF - if compile_prog "$CPU_CFLAGS -Werror -fsanitize=undefined" ""; then - have_ubsan=yes - fi - - if check_include "sanitizer/asan_interface.h" ; then - have_asan_iface_h=yes - fi - - cat > $TMPC << EOF -#include -int main(void) { - __sanitizer_start_switch_fiber(0, 0, 0); - return 0; -} -EOF - if compile_prog "$CPU_CFLAGS -Werror -fsanitize=address" "" ; then - have_asan_iface_fiber=yes - fi -fi - -# Thread sanitizer is, for now, much noisier than the other sanitizers; -# keep it separate until that is not the case. -if test "$tsan" = "yes" && test "$sanitizers" = "yes"; then - error_exit "TSAN is not supported with other sanitiziers." -fi -have_tsan=no -have_tsan_iface_fiber=no -if test "$tsan" = "yes" ; then - write_c_skeleton - if compile_prog "$CPU_CFLAGS -Werror -fsanitize=thread" "" ; then - have_tsan=yes - fi - cat > $TMPC << EOF -#include -int main(void) { - __tsan_create_fiber(0); - return 0; -} -EOF - if compile_prog "$CPU_CFLAGS -Werror -fsanitize=thread" "" ; then - have_tsan_iface_fiber=yes - fi -fi - ########################################## # functions to probe cross compilers @@ -2139,42 +2036,6 @@ case "$vfio_user_server" in ;; esac -########################################## -# End of CC checks -# After here, no more $cc or $ld runs - -write_c_skeleton - -if test "$fortify_source" = "yes" ; then - QEMU_CFLAGS="-U_FORTIFY_SOURCE -D_FORTIFY_SOURCE=2 $QEMU_CFLAGS" -fi - -if test "$have_asan" = "yes"; then - QEMU_CFLAGS="-fsanitize=address $QEMU_CFLAGS" - QEMU_LDFLAGS="-fsanitize=address $QEMU_LDFLAGS" - if test "$have_asan_iface_h" = "no" ; then - echo "ASAN build enabled, but ASAN header missing." \ - "Without code annotation, the report may be inferior." - elif test "$have_asan_iface_fiber" = "no" ; then - echo "ASAN build enabled, but ASAN header is too old." \ - "Without code annotation, the report may be inferior." - fi -fi -if test "$have_tsan" = "yes" ; then - if test "$have_tsan_iface_fiber" = "yes" ; then - QEMU_CFLAGS="-fsanitize=thread $QEMU_CFLAGS" - QEMU_LDFLAGS="-fsanitize=thread $QEMU_LDFLAGS" - else - error_exit "Cannot enable TSAN due to missing fiber annotation interface." - fi -elif test "$tsan" = "yes" ; then - error_exit "Cannot enable TSAN due to missing sanitize thread interface." -fi -if test "$have_ubsan" = "yes"; then - QEMU_CFLAGS="-fsanitize=undefined $QEMU_CFLAGS" - QEMU_LDFLAGS="-fsanitize=undefined $QEMU_LDFLAGS" -fi - ####################################### # cross-compiled firmware targets @@ -2299,14 +2160,6 @@ fi echo "CONFIG_COROUTINE_BACKEND=$coroutine" >> $config_host_mak -if test "$have_asan_iface_fiber" = "yes" ; then - echo "CONFIG_ASAN_IFACE_FIBER=y" >> $config_host_mak -fi - -if test "$have_tsan" = "yes" && test "$have_tsan_iface_fiber" = "yes" ; then - echo "CONFIG_TSAN=y" >> $config_host_mak -fi - if test "$plugins" = "yes" ; then echo "CONFIG_PLUGIN=y" >> $config_host_mak fi diff --git a/docs/devel/build-system.rst b/docs/devel/build-system.rst index 66cfe7b8bdc8..4a733fc0a747 100644 --- a/docs/devel/build-system.rst +++ b/docs/devel/build-system.rst @@ -91,10 +91,6 @@ developers in checking for system features: ``check_define $NAME`` Determine if the macro $NAME is defined by the system C compiler -``check_include $NAME`` - Determine if the include $NAME file is available to the system C - compiler. The replacement in Meson is ``cc.has_header()``. - ``write_c_skeleton`` Write a minimal C program main() function to the temporary file indicated by $TMPC diff --git a/meson.build b/meson.build index f162ed4e335f..b4d2a53b6c36 100644 --- a/meson.build +++ b/meson.build @@ -211,6 +211,35 @@ if get_option('prefer_static') qemu_ldflags += get_option('b_pie') ? '-static-pie' : '-static' endif +if get_option('sanitizers') + if cc.has_argument('-fsanitize=address') + qemu_cflags = ['-fsanitize=address'] + qemu_cflags + qemu_ldflags = ['-fsanitize=address'] + qemu_ldflags + endif + + # Detect static linking issue with ubsan - https://gcc.gnu.org/bugzilla/show_bug.cgi?id=84285 + if cc.links('int main(int argc, char **argv) { return argc + 1; }', + args: [qemu_ldflags, '-fsanitize=undefined']) + qemu_cflags = ['-fsanitize=undefined'] + qemu_cflags + qemu_ldflags = ['-fsanitize=undefined'] + qemu_ldflags + endif +endif + +# Thread sanitizer is, for now, much noisier than the other sanitizers; +# keep it separate until that is not the case. +if get_option('tsan') + if get_option('sanitizers') + error('TSAN is not supported with other sanitizers') + endif + if not cc.has_function('__tsan_create_fiber', + args: '-fsanitize=thread', + prefix: '#include ') + error('Cannot enable TSAN due to missing fiber annotation interface') + endif + qemu_cflags = ['-fsanitize=thread'] + qemu_cflags + qemu_ldflags = ['-fsanitize=thread'] + qemu_ldflags +endif + # Detect support for PT_GNU_RELRO + DT_BIND_NOW. # The combination is known as "full relro", because .got.plt is read-only too. qemu_ldflags += cc.get_supported_link_arguments('-Wl,-z,relro', '-Wl,-z,now') @@ -221,7 +250,7 @@ if targetos == 'windows' endif # Exclude --warn-common with TSan to suppress warnings from the TSan libraries. -if targetos != 'sunos' and not config_host.has_key('CONFIG_TSAN') +if targetos != 'sunos' and not get_option('tsan') qemu_ldflags += cc.get_supported_link_arguments('-Wl,--warn-common') endif @@ -284,6 +313,16 @@ if 'cpp' in all_languages endif endif +# clang does not support glibc + FORTIFY_SOURCE (is it still true?) +if get_option('optimization') != '0' and targetos == 'linux' + if cc.get_id() == 'gcc' + qemu_cflags += ['-U_FORTIFY_SOURCE', '-D_FORTIFY_SOURCE=2'] + endif + if 'cpp' in all_languages and cxx.get_id() == 'gcc' + qemu_cxxflags += ['-U_FORTIFY_SOURCE', '-D_FORTIFY_SOURCE=2'] + endif +endif + add_project_arguments(qemu_cflags, native: false, language: 'c') add_project_arguments(qemu_cxxflags, native: false, language: 'cpp') add_project_arguments(qemu_objcflags, native: false, language: 'objc') @@ -1915,6 +1954,7 @@ if seccomp.found() endif config_host_data.set('CONFIG_SNAPPY', snappy.found()) config_host_data.set('CONFIG_TPM', have_tpm) +config_host_data.set('CONFIG_TSAN', get_option('tsan')) config_host_data.set('CONFIG_USB_LIBUSB', libusb.found()) config_host_data.set('CONFIG_VDE', vde.found()) config_host_data.set('CONFIG_VHOST_NET', have_vhost_net) @@ -2046,6 +2086,18 @@ if rdma.found() prefix: '#include ')) endif +have_asan_fiber = false +if get_option('sanitizers') and \ + not cc.has_function('__sanitizer_start_switch_fiber', + args: '-fsanitize=address', + prefix: '#include ') + warning('Missing ASAN due to missing fiber annotation interface') + warning('Without code annotation, the report may be inferior.') +else + have_asan_fiber = true +endif +config_host_data.set('CONFIG_ASAN_IFACE_FIBER', have_asan_fiber) + # has_header_symbol config_host_data.set('CONFIG_BLKZONED', cc.has_header_symbol('linux/blkzoned.h', 'BLKOPENZONE')) @@ -3877,7 +3929,7 @@ else endif summary_info += {'gprof': gprof_info} summary_info += {'gcov': get_option('b_coverage')} -summary_info += {'thread sanitizer': config_host.has_key('CONFIG_TSAN')} +summary_info += {'thread sanitizer': get_option('tsan')} summary_info += {'CFI support': get_option('cfi')} if get_option('cfi') summary_info += {'CFI debug support': get_option('cfi_debug')} diff --git a/meson_options.txt b/meson_options.txt index a1240e0b2a59..a15d434267f9 100644 --- a/meson_options.txt +++ b/meson_options.txt @@ -82,6 +82,10 @@ option('tcg', type: 'feature', value: 'enabled', description: 'TCG support') option('tcg_interpreter', type: 'boolean', value: false, description: 'TCG with bytecode interpreter (slow)') +option('sanitizers', type: 'boolean', value: false, + description: 'enable default sanitizers') +option('tsan', type: 'boolean', value: false, + description: 'enable thread sanitizer') option('cfi', type: 'boolean', value: false, description: 'Control-Flow Integrity (CFI)') option('cfi_debug', type: 'boolean', value: false, diff --git a/scripts/meson-buildoptions.sh b/scripts/meson-buildoptions.sh index 43301864e395..b79a14b6948c 100644 --- a/scripts/meson-buildoptions.sh +++ b/scripts/meson-buildoptions.sh @@ -41,11 +41,13 @@ meson_options_help() { printf "%s\n" ' --enable-profiler profiler support' printf "%s\n" ' --enable-rng-none dummy RNG, avoid using /dev/(u)random and' printf "%s\n" ' getrandom()' + printf "%s\n" ' --enable-sanitizers enable default sanitizers' printf "%s\n" ' --enable-strip Strip targets on install' printf "%s\n" ' --enable-tcg-interpreter TCG with bytecode interpreter (slow)' printf "%s\n" ' --enable-trace-backends=CHOICES' printf "%s\n" ' Set available tracing backends [log] (choices:' printf "%s\n" ' dtrace/ftrace/log/nop/simple/syslog/ust)' + printf "%s\n" ' --enable-tsan enable thread sanitizer' printf "%s\n" ' --firmwarepath=VALUES search PATH for firmware files [share/qemu-' printf "%s\n" ' firmware]' printf "%s\n" ' --iasl=VALUE Path to ACPI disassembler' @@ -406,6 +408,8 @@ _meson_option_parse() { --disable-replication) printf "%s" -Dreplication=disabled ;; --enable-rng-none) printf "%s" -Drng_none=true ;; --disable-rng-none) printf "%s" -Drng_none=false ;; + --enable-sanitizers) printf "%s" -Dsanitizers=true ;; + --disable-sanitizers) printf "%s" -Dsanitizers=false ;; --enable-sdl) printf "%s" -Dsdl=enabled ;; --disable-sdl) printf "%s" -Dsdl=disabled ;; --enable-sdl-image) printf "%s" -Dsdl_image=enabled ;; @@ -444,6 +448,8 @@ _meson_option_parse() { --disable-tpm) printf "%s" -Dtpm=disabled ;; --enable-trace-backends=*) quote_sh "-Dtrace_backends=$2" ;; --with-trace-file=*) quote_sh "-Dtrace_file=$2" ;; + --enable-tsan) printf "%s" -Dtsan=true ;; + --disable-tsan) printf "%s" -Dtsan=false ;; --enable-u2f) printf "%s" -Du2f=enabled ;; --disable-u2f) printf "%s" -Du2f=disabled ;; --enable-usb-redir) printf "%s" -Dusb_redir=enabled ;; diff --git a/tests/qemu-iotests/meson.build b/tests/qemu-iotests/meson.build index 9735071a295b..44761e1e4d3e 100644 --- a/tests/qemu-iotests/meson.build +++ b/tests/qemu-iotests/meson.build @@ -2,7 +2,7 @@ if not have_tools or targetos == 'windows' or get_option('gprof') subdir_done() endif -foreach cflag: config_host['QEMU_CFLAGS'].split() +foreach cflag: qemu_ldflags if cflag.startswith('-fsanitize') and \ not cflag.contains('safe-stack') and not cflag.contains('cfi-icall') message('Sanitizers are enabled ==> Disabled the qemu-iotests.') diff --git a/tests/unit/meson.build b/tests/unit/meson.build index 3bc78d8660a4..48ae66011b17 100644 --- a/tests/unit/meson.build +++ b/tests/unit/meson.build @@ -147,7 +147,7 @@ if have_system # Some tests: test-char, test-qdev-global-props, and test-qga, # are not runnable under TSan due to a known issue. # https://github.com/google/sanitizers/issues/1116 - if 'CONFIG_TSAN' not in config_host + if not get_option('tsan') if 'CONFIG_POSIX' in config_host tests += { 'test-char': ['socket-helpers.c', qom, io, chardev] From patchwork Wed May 17 17:45:08 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Paolo Bonzini X-Patchwork-Id: 1782885 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=nongnu.org (client-ip=209.51.188.17; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=) Authentication-Results: legolas.ozlabs.org; dkim=pass (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=emB90You; dkim-atps=neutral Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-ECDSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4QM1FH6l07z20dn for ; Thu, 18 May 2023 04:03:47 +1000 (AEST) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1pzLFL-0002st-AY; Wed, 17 May 2023 13:47:35 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pzLEq-0001x3-S7 for qemu-devel@nongnu.org; Wed, 17 May 2023 13:47:06 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.129.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pzLEn-0004SF-UP for qemu-devel@nongnu.org; Wed, 17 May 2023 13:46:59 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1684345617; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=vWsBITqXH0Hf/HJAkrmR2QoEA1Mw0DfApnEDWhv8x/I=; b=emB90You2k2oLfMzr48EOHthbHuxedZ13YRNkWBPqODt3jlrhWQLPkxBnosE7uxh075eik YALPZQ6SG5BM1G8YiG4JGtI/dfx+uaPj5wXEvXdS2FXse40FlFfmeGEsbsr6w4ltBRNXUB g/qza3xQWjxkMmYwxicf7CtQHyTsdOk= Received: from mail-wr1-f69.google.com (mail-wr1-f69.google.com [209.85.221.69]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-605-39GbaHIPM1uqaFnmiEgVdQ-1; Wed, 17 May 2023 13:46:56 -0400 X-MC-Unique: 39GbaHIPM1uqaFnmiEgVdQ-1 Received: by mail-wr1-f69.google.com with SMTP id ffacd0b85a97d-30479b764f9so451648f8f.0 for ; Wed, 17 May 2023 10:46:55 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1684345614; x=1686937614; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=vWsBITqXH0Hf/HJAkrmR2QoEA1Mw0DfApnEDWhv8x/I=; b=AoScm6U+rD45OMx5Q68NZmm6kQT8UcJU6VukRmEla0XHfamEaswlKTW6O3kl8heCRY 7SfTUyq6Dd8XoyXMlNZFV4ljJAR39DnIAnDAKci3owlqEPOD6IS4UDTjaOa6928esnmr XcqQseffecUOoFn2Soe5axyoGWZ+SRPo0M1XkkvUeSCBWaW+Np37uQd5BW6J7boTyqHo 3YB6kO0N7WAd6jfdrOKtZPO57YGgjJMBw8eTv5aeR0GgEYIrpsZ4mIWOFBVkZaL3jNOS a9k/NetKQSwE5+f/3WM2s0PkGJv7DBw8dByVYE6RSmjTubKb/KC1U0+VqSnbz3ucQu1g N1IA== X-Gm-Message-State: AC+VfDwToA8G0STha85I4iYh8WggUlOTGNzeivicNTODcetrtk874GCO BKPx33xZgwmYK5eU/jEGx6dQhsfD+UHOUzrJQGg+0BtVNdR+vqM4MShvkhgFmHzE0of+mnXNj1V hU3B0g3z6rVuF6/3CoI9v9+RS38a8z9xlCnEIDMA9mydVE9+ac6MP6RUHs5XtnIXq5SiA82O45O U= X-Received: by 2002:a5d:65c5:0:b0:306:35fa:202b with SMTP id e5-20020a5d65c5000000b0030635fa202bmr1223286wrw.6.1684345614366; Wed, 17 May 2023 10:46:54 -0700 (PDT) X-Google-Smtp-Source: ACHHUZ61F4NqQdExSbhV8yewi5q1b4pYc2g/fPwRFTE4DQxQe9veXOXQcb6QWpzVHmgLcY0xgLjQSQ== X-Received: by 2002:a5d:65c5:0:b0:306:35fa:202b with SMTP id e5-20020a5d65c5000000b0030635fa202bmr1223276wrw.6.1684345614036; Wed, 17 May 2023 10:46:54 -0700 (PDT) Received: from [192.168.10.118] ([2001:b07:6468:f312:9af8:e5f5:7516:fa89]) by smtp.gmail.com with ESMTPSA id q7-20020adfcd87000000b003062b2c5255sm3400498wrj.40.2023.05.17.10.46.53 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 17 May 2023 10:46:53 -0700 (PDT) From: Paolo Bonzini To: qemu-devel@nongnu.org Cc: =?utf-8?q?Philippe_Mathieu-Daud=C3=A9?= Subject: [PULL 56/68] build: move SafeStack tests to meson Date: Wed, 17 May 2023 19:45:08 +0200 Message-Id: <20230517174520.887405-57-pbonzini@redhat.com> X-Mailer: git-send-email 2.40.1 In-Reply-To: <20230517174520.887405-1-pbonzini@redhat.com> References: <20230517174520.887405-1-pbonzini@redhat.com> MIME-Version: 1.0 Received-SPF: pass client-ip=170.10.129.124; envelope-from=pbonzini@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org This disables the old behavior of detecting SafeStack from environment CFLAGS. SafeStack is now enabled purely based on the configure arguments. Reviewed-by: Philippe Mathieu-DaudĂ© Signed-off-by: Paolo Bonzini --- configure | 73 ----------------------------------- meson.build | 26 +++++++++++++ meson_options.txt | 2 + scripts/meson-buildoptions.sh | 4 ++ 4 files changed, 32 insertions(+), 73 deletions(-) diff --git a/configure b/configure index 34b26c45cf39..1cde6f698161 100755 --- a/configure +++ b/configure @@ -227,7 +227,6 @@ cross_compile="no" cross_prefix="" host_cc="cc" stack_protector="" -safe_stack="" use_containers="yes" gdb_bin=$(command -v "gdb-multiarch" || command -v "gdb") gdb_arches="" @@ -830,10 +829,6 @@ for opt do ;; --disable-stack-protector) stack_protector="no" ;; - --enable-safe-stack) safe_stack="yes" - ;; - --disable-safe-stack) safe_stack="no" - ;; --enable-cfi) cfi="true"; meson_option_add -Db_lto=true @@ -1024,8 +1019,6 @@ cat << EOF pie Position Independent Executables debug-tcg TCG debugging (default is disabled) debug-info debugging information - safe-stack SafeStack Stack Smash Protection. Depends on - clang/llvm and requires coroutine backend ucontext. NOTE: The object files are built at the place where configure is launched EOF @@ -1521,68 +1514,6 @@ else esac fi -################################################## -# SafeStack - - -if test "$safe_stack" = "yes"; then -cat > $TMPC << EOF -int main(void) -{ -#if ! __has_feature(safe_stack) -#error SafeStack Disabled -#endif - return 0; -} -EOF - flag="-fsanitize=safe-stack" - # Check that safe-stack is supported and enabled. - if compile_prog "-Werror $flag" "$flag"; then - # Flag needed both at compilation and at linking - QEMU_CFLAGS="$QEMU_CFLAGS $flag" - QEMU_LDFLAGS="$QEMU_LDFLAGS $flag" - else - error_exit "SafeStack not supported by your compiler" - fi - if test "$coroutine" != "ucontext"; then - error_exit "SafeStack is only supported by the coroutine backend ucontext" - fi -else -cat > $TMPC << EOF -int main(void) -{ -#if defined(__has_feature) -#if __has_feature(safe_stack) -#error SafeStack Enabled -#endif -#endif - return 0; -} -EOF -if test "$safe_stack" = "no"; then - # Make sure that safe-stack is disabled - if ! compile_prog "-Werror" ""; then - # SafeStack was already enabled, try to explicitly remove the feature - flag="-fno-sanitize=safe-stack" - if ! compile_prog "-Werror $flag" "$flag"; then - error_exit "Configure cannot disable SafeStack" - fi - QEMU_CFLAGS="$QEMU_CFLAGS $flag" - QEMU_LDFLAGS="$QEMU_LDFLAGS $flag" - fi -else # "$safe_stack" = "" - # Set safe_stack to yes or no based on pre-existing flags - if compile_prog "-Werror" ""; then - safe_stack="no" - else - safe_stack="yes" - if test "$coroutine" != "ucontext"; then - error_exit "SafeStack is only supported by the coroutine backend ucontext" - fi - fi -fi -fi - ######################################## # check if ccache is interfering with # semantic analysis of macros @@ -2242,10 +2173,6 @@ if test "$ccache_cpp2" = "yes"; then echo "export CCACHE_CPP2=y" >> $config_host_mak fi -if test "$safe_stack" = "yes"; then - echo "CONFIG_SAFESTACK=y" >> $config_host_mak -fi - # tests/tcg configuration (config_host_mak=tests/tcg/config-host.mak mkdir -p tests/tcg diff --git a/meson.build b/meson.build index b4d2a53b6c36..b83b3c937d8f 100644 --- a/meson.build +++ b/meson.build @@ -211,6 +211,31 @@ if get_option('prefer_static') qemu_ldflags += get_option('b_pie') ? '-static-pie' : '-static' endif +# Compiles if SafeStack *not* enabled +safe_stack_probe = ''' + int main(void) + { + #if defined(__has_feature) + #if __has_feature(safe_stack) + #error SafeStack Enabled + #endif + #endif + return 0; + }''' +if get_option('safe_stack') != not cc.compiles(safe_stack_probe) + safe_stack_arg = get_option('safe_stack') ? '-fsanitize=safe-stack' : '-fno-sanitize=safe-stack' + if get_option('safe_stack') != not cc.compiles(safe_stack_probe, args: safe_stack_arg) + error(get_option('safe_stack') \ + ? 'SafeStack not supported by your compiler' \ + : 'Cannot disable SafeStack') + endif + qemu_cflags += safe_stack_arg + qemu_ldflags += safe_stack_arg +endif +if get_option('safe_stack') and config_host['CONFIG_COROUTINE_BACKEND'] != 'ucontext' + error('SafeStack is only supported with the ucontext coroutine backend') +endif + if get_option('sanitizers') if cc.has_argument('-fsanitize=address') qemu_cflags = ['-fsanitize=address'] + qemu_cflags @@ -1946,6 +1971,7 @@ config_host_data.set('CONFIG_OPENGL', opengl.found()) config_host_data.set('CONFIG_PROFILER', get_option('profiler')) config_host_data.set('CONFIG_RBD', rbd.found()) config_host_data.set('CONFIG_RDMA', rdma.found()) +config_host_data.set('CONFIG_SAFESTACK', get_option('safe_stack')) config_host_data.set('CONFIG_SDL', sdl.found()) config_host_data.set('CONFIG_SDL_IMAGE', sdl_image.found()) config_host_data.set('CONFIG_SECCOMP', seccomp.found()) diff --git a/meson_options.txt b/meson_options.txt index a15d434267f9..89bc06b561e6 100644 --- a/meson_options.txt +++ b/meson_options.txt @@ -82,6 +82,8 @@ option('tcg', type: 'feature', value: 'enabled', description: 'TCG support') option('tcg_interpreter', type: 'boolean', value: false, description: 'TCG with bytecode interpreter (slow)') +option('safe_stack', type: 'boolean', value: false, + description: 'SafeStack Stack Smash Protection (requires clang/llvm and coroutine backend ucontext)') option('sanitizers', type: 'boolean', value: false, description: 'enable default sanitizers') option('tsan', type: 'boolean', value: false, diff --git a/scripts/meson-buildoptions.sh b/scripts/meson-buildoptions.sh index b79a14b6948c..35dbfff2387a 100644 --- a/scripts/meson-buildoptions.sh +++ b/scripts/meson-buildoptions.sh @@ -41,6 +41,8 @@ meson_options_help() { printf "%s\n" ' --enable-profiler profiler support' printf "%s\n" ' --enable-rng-none dummy RNG, avoid using /dev/(u)random and' printf "%s\n" ' getrandom()' + printf "%s\n" ' --enable-safe-stack SafeStack Stack Smash Protection (requires' + printf "%s\n" ' clang/llvm and coroutine backend ucontext)' printf "%s\n" ' --enable-sanitizers enable default sanitizers' printf "%s\n" ' --enable-strip Strip targets on install' printf "%s\n" ' --enable-tcg-interpreter TCG with bytecode interpreter (slow)' @@ -408,6 +410,8 @@ _meson_option_parse() { --disable-replication) printf "%s" -Dreplication=disabled ;; --enable-rng-none) printf "%s" -Drng_none=true ;; --disable-rng-none) printf "%s" -Drng_none=false ;; + --enable-safe-stack) printf "%s" -Dsafe_stack=true ;; + --disable-safe-stack) printf "%s" -Dsafe_stack=false ;; --enable-sanitizers) printf "%s" -Dsanitizers=true ;; --disable-sanitizers) printf "%s" -Dsanitizers=false ;; --enable-sdl) printf "%s" -Dsdl=enabled ;; From patchwork Wed May 17 17:45:09 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Paolo Bonzini X-Patchwork-Id: 1782865 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=nongnu.org (client-ip=209.51.188.17; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=) Authentication-Results: legolas.ozlabs.org; dkim=pass (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=bIWHmLZj; dkim-atps=neutral Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-ECDSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4QM14w0Yzzz20dX for ; Thu, 18 May 2023 03:56:32 +1000 (AEST) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1pzLFx-0003Fy-1m; Wed, 17 May 2023 13:48:09 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pzLEs-0001xF-R7 for qemu-devel@nongnu.org; Wed, 17 May 2023 13:47:08 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.129.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pzLEq-0004UA-QO for qemu-devel@nongnu.org; Wed, 17 May 2023 13:47:02 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1684345619; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=0G45ZYVI2xWZEBapYoD8PtloM4cggEZlrWBmcULTD2Q=; b=bIWHmLZjYZLNY8ur6b+dDtD5ruGMM7RSvk3cM0GQTM8eWm/7/HPcZ65Lt1VH45Ub+X6Tal iFfofJCC2xcW5Jj5QXCWvTIhpDFAv/KtKpempBeq/59e4aeG91tEMD+hEDdn+u83eFEeGu QHm31fvSWgr0+WroRRNNzr6t7Au2qds= Received: from mail-wm1-f72.google.com (mail-wm1-f72.google.com [209.85.128.72]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-19-oq0pulJdPnOj9xnBV8mHAQ-1; Wed, 17 May 2023 13:46:58 -0400 X-MC-Unique: oq0pulJdPnOj9xnBV8mHAQ-1 Received: by mail-wm1-f72.google.com with SMTP id 5b1f17b1804b1-3f42b36733aso4441145e9.3 for ; Wed, 17 May 2023 10:46:57 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1684345616; x=1686937616; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=0G45ZYVI2xWZEBapYoD8PtloM4cggEZlrWBmcULTD2Q=; b=jnrOTP4Yr08qBUmVgOuA+LykWgeeJRy+11A08/bLLrGlXw1L/OxWdORq8pq5BrdU6g G53vyzrkZTX4eI5KRc9J02De/SWvoxT/oFEkQLeD0vCz4hQMHsKYoQl2KNPfB6ozmov7 AdzoYeJFVQakGXlNZDHdES3IXZTNE/WMJMDYwVgeYUSwfBpxQYsCfafpndEmabP1XlDp Ixa1K5WGBhKg5gToURPzONvHWOwfrPAECzt5idSyOho1NKt38aOMKg+TEuw9dJUgnIIR /3gOLJ+Er/HmYh9LaU6EWs+8+vl/iyeyELCgTBm8KtWuBHbSHlf6DrRHQl1WgyPXfAl8 K+Mg== X-Gm-Message-State: AC+VfDxF1daGoy8fLiFo+2fo0slHZ2CFdDcLCfe3Qqe1aeQBqM+bNqvB +eOV74DlXfo/i3wtjDfht0jM1y9Y4u4hN39tnKGl290srAQLQCTc5PL8xJPx/Cz0k+lq4/vqB2h RYH8TXMItSpZb6UpFdRBpKbQQN+Psf+eXJViY8+ryAHdEiVq24eET2PUYUDdd2KeSClv4UE8fIy w= X-Received: by 2002:a7b:c4c3:0:b0:3f1:8085:910e with SMTP id g3-20020a7bc4c3000000b003f18085910emr29122415wmk.9.1684345616220; Wed, 17 May 2023 10:46:56 -0700 (PDT) X-Google-Smtp-Source: ACHHUZ4i5WLusndBoffSHjsNVt4CkV2NGDdsyjKILJ+F2MQ5+DzRt6VjBLxCJUapkRGz3JrEhpb9ng== X-Received: by 2002:a7b:c4c3:0:b0:3f1:8085:910e with SMTP id g3-20020a7bc4c3000000b003f18085910emr29122404wmk.9.1684345615890; Wed, 17 May 2023 10:46:55 -0700 (PDT) Received: from [192.168.10.118] ([2001:b07:6468:f312:9af8:e5f5:7516:fa89]) by smtp.gmail.com with ESMTPSA id r4-20020a056000014400b002f103ca90cdsm3384327wrx.101.2023.05.17.10.46.54 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 17 May 2023 10:46:55 -0700 (PDT) From: Paolo Bonzini To: qemu-devel@nongnu.org Cc: =?utf-8?q?Marc-Andr=C3=A9_Lureau?= Subject: [PULL 57/68] build: move coroutine backend selection to meson Date: Wed, 17 May 2023 19:45:09 +0200 Message-Id: <20230517174520.887405-58-pbonzini@redhat.com> X-Mailer: git-send-email 2.40.1 In-Reply-To: <20230517174520.887405-1-pbonzini@redhat.com> References: <20230517174520.887405-1-pbonzini@redhat.com> MIME-Version: 1.0 Received-SPF: pass client-ip=170.10.129.124; envelope-from=pbonzini@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org To simplify the code, rename coroutine-win32.c to match the option passed to configure. Reviewed-by: Marc-AndrĂ© Lureau Signed-off-by: Paolo Bonzini --- configure | 62 ------------------- meson.build | 32 +++++++++- meson_options.txt | 3 + scripts/meson-buildoptions.py | 1 + scripts/meson-buildoptions.sh | 3 + ...{coroutine-win32.c => coroutine-windows.c} | 0 util/meson.build | 2 +- 7 files changed, 38 insertions(+), 65 deletions(-) rename util/{coroutine-win32.c => coroutine-windows.c} (100%) diff --git a/configure b/configure index 1cde6f698161..470ed9f8c201 100755 --- a/configure +++ b/configure @@ -282,7 +282,6 @@ softmmu="yes" linux_user="" bsd_user="" pie="" -coroutine="" plugins="$default_feature" ninja="" bindir="bin" @@ -843,8 +842,6 @@ for opt do ;; --enable-fdt=*) fdt="$optarg" ;; - --with-coroutine=*) coroutine="$optarg" - ;; --with-git=*) git="$optarg" ;; --with-git-submodules=*) @@ -1003,8 +1000,6 @@ Advanced options (experts only): --disable-werror disable compilation abort on warning --disable-stack-protector disable compiler-provided stack protection --cpu=CPU Build for host CPU [$cpu] - --with-coroutine=BACKEND coroutine backend. Supported options: - ucontext, sigaltstack, windows --enable-plugins enable plugins via shared library loading --disable-containers don't use containers for cross-building @@ -1459,61 +1454,6 @@ case "$fdt" in ;; esac -########################################## -# check and set a backend for coroutine - -# We prefer ucontext, but it's not always possible. The fallback -# is sigcontext. On Windows the only valid backend is the Windows -# specific one. - -ucontext_works=no -if test "$darwin" != "yes"; then - cat > $TMPC << EOF -#include -#ifdef __stub_makecontext -#error Ignoring glibc stub makecontext which will always fail -#endif -int main(void) { makecontext(0, 0, 0); return 0; } -EOF - if compile_prog "" "" ; then - ucontext_works=yes - fi -fi - -if test "$coroutine" = ""; then - if test "$mingw32" = "yes"; then - coroutine=win32 - elif test "$ucontext_works" = "yes"; then - coroutine=ucontext - else - coroutine=sigaltstack - fi -else - case $coroutine in - windows) - if test "$mingw32" != "yes"; then - error_exit "'windows' coroutine backend only valid for Windows" - fi - # Unfortunately the user visible backend name doesn't match the - # coroutine-*.c filename for this case, so we have to adjust it here. - coroutine=win32 - ;; - ucontext) - if test "$ucontext_works" != "yes"; then - error_exit "'ucontext' backend requested but makecontext not available" - fi - ;; - sigaltstack) - if test "$mingw32" = "yes"; then - error_exit "only the 'windows' coroutine backend is valid for Windows" - fi - ;; - *) - error_exit "unknown coroutine backend $coroutine" - ;; - esac -fi - ######################################## # check if ccache is interfering with # semantic analysis of macros @@ -2089,8 +2029,6 @@ if [ "$bsd" = "yes" ] ; then echo "CONFIG_BSD=y" >> $config_host_mak fi -echo "CONFIG_COROUTINE_BACKEND=$coroutine" >> $config_host_mak - if test "$plugins" = "yes" ; then echo "CONFIG_PLUGIN=y" >> $config_host_mak fi diff --git a/meson.build b/meson.build index b83b3c937d8f..f410afe8b044 100644 --- a/meson.build +++ b/meson.build @@ -211,6 +211,34 @@ if get_option('prefer_static') qemu_ldflags += get_option('b_pie') ? '-static-pie' : '-static' endif +coroutine_backend = get_option('coroutine_backend') +ucontext_probe = ''' + #include + #ifdef __stub_makecontext + #error Ignoring glibc stub makecontext which will always fail + #endif + int main(void) { makecontext(0, 0, 0); return 0; }''' + +# On Windows the only valid backend is the Windows specific one. +# For POSIX prefer ucontext, but it's not always possible. The fallback +# is sigcontext. +supported_backends = [] +if targetos == 'windows' + supported_backends += ['windows'] +else + if targetos != 'darwin' and cc.links(ucontext_probe) + supported_backends += ['ucontext'] + endif + supported_backends += ['sigaltstack'] +endif + +if coroutine_backend == 'auto' + coroutine_backend = supported_backends[0] +elif coroutine_backend not in supported_backends + error('"@0@" backend requested but not available. Available backends: @1@' \ + .format(coroutine_backend, ', '.join(supported_backends))) +endif + # Compiles if SafeStack *not* enabled safe_stack_probe = ''' int main(void) @@ -232,7 +260,7 @@ if get_option('safe_stack') != not cc.compiles(safe_stack_probe) qemu_cflags += safe_stack_arg qemu_ldflags += safe_stack_arg endif -if get_option('safe_stack') and config_host['CONFIG_COROUTINE_BACKEND'] != 'ucontext' +if get_option('safe_stack') and coroutine_backend != 'ucontext' error('SafeStack is only supported with the ucontext coroutine backend') endif @@ -4016,7 +4044,7 @@ summary(summary_info, bool_yn: true, section: 'Targets and accelerators') # Block layer summary_info = {} -summary_info += {'coroutine backend': config_host['CONFIG_COROUTINE_BACKEND']} +summary_info += {'coroutine backend': coroutine_backend} summary_info += {'coroutine pool': have_coroutine_pool} if have_block summary_info += {'Block whitelist (rw)': get_option('block_drv_rw_whitelist')} diff --git a/meson_options.txt b/meson_options.txt index 89bc06b561e6..69de30c83db0 100644 --- a/meson_options.txt +++ b/meson_options.txt @@ -31,6 +31,9 @@ option('fuzzing_engine', type : 'string', value : '', description: 'fuzzing engine library for OSS-Fuzz') option('trace_file', type: 'string', value: 'trace', description: 'Trace file prefix for simple backend') +option('coroutine_backend', type: 'combo', + choices: ['ucontext', 'sigaltstack', 'windows', 'auto'], + value: 'auto', description: 'coroutine backend to use') # Everything else can be set via --enable/--disable-* option # on the configure script command line. After adding an option diff --git a/scripts/meson-buildoptions.py b/scripts/meson-buildoptions.py index a04dcc70a5b7..4c7f13fdfc40 100755 --- a/scripts/meson-buildoptions.py +++ b/scripts/meson-buildoptions.py @@ -35,6 +35,7 @@ OPTION_NAMES = { "b_coverage": "gcov", "b_lto": "lto", + "coroutine_backend": "with-coroutine", "malloc": "enable-malloc", "pkgversion": "with-pkgversion", "qemu_firmwarepath": "firmwarepath", diff --git a/scripts/meson-buildoptions.sh b/scripts/meson-buildoptions.sh index 35dbfff2387a..032fcea948d4 100644 --- a/scripts/meson-buildoptions.sh +++ b/scripts/meson-buildoptions.sh @@ -64,6 +64,8 @@ meson_options_help() { printf "%s\n" ' --sysconfdir=VALUE Sysconf data directory [etc]' printf "%s\n" ' --tls-priority=VALUE Default TLS protocol/cipher priority string' printf "%s\n" ' [NORMAL]' + printf "%s\n" ' --with-coroutine=CHOICE coroutine backend to use (choices:' + printf "%s\n" ' auto/sigaltstack/ucontext/windows)' printf "%s\n" ' --with-pkgversion=VALUE use specified string as sub-version of the' printf "%s\n" ' package' printf "%s\n" ' --with-trace-file=VALUE Trace file prefix for simple backend [trace]' @@ -248,6 +250,7 @@ _meson_option_parse() { --disable-cocoa) printf "%s" -Dcocoa=disabled ;; --enable-coreaudio) printf "%s" -Dcoreaudio=enabled ;; --disable-coreaudio) printf "%s" -Dcoreaudio=disabled ;; + --with-coroutine=*) quote_sh "-Dcoroutine_backend=$2" ;; --enable-coroutine-pool) printf "%s" -Dcoroutine_pool=true ;; --disable-coroutine-pool) printf "%s" -Dcoroutine_pool=false ;; --enable-crypto-afalg) printf "%s" -Dcrypto_afalg=enabled ;; diff --git a/util/coroutine-win32.c b/util/coroutine-windows.c similarity index 100% rename from util/coroutine-win32.c rename to util/coroutine-windows.c diff --git a/util/meson.build b/util/meson.build index 2cb103fc727e..e1f1c39e1081 100644 --- a/util/meson.build +++ b/util/meson.build @@ -78,7 +78,7 @@ if have_block or have_ga util_ss.add(files('base64.c')) util_ss.add(files('main-loop.c')) util_ss.add(files('qemu-coroutine.c', 'qemu-coroutine-lock.c', 'qemu-coroutine-io.c')) - util_ss.add(files('coroutine-@0@.c'.format(config_host['CONFIG_COROUTINE_BACKEND']))) + util_ss.add(files(f'coroutine-@coroutine_backend@.c')) util_ss.add(files('thread-pool.c', 'qemu-timer.c')) util_ss.add(files('qemu-sockets.c')) endif From patchwork Wed May 17 17:45:10 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Paolo Bonzini X-Patchwork-Id: 1782864 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=nongnu.org (client-ip=209.51.188.17; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=) Authentication-Results: legolas.ozlabs.org; dkim=pass (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=DekeseMj; dkim-atps=neutral Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-ECDSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4QM14s2bZhz20dX for ; Thu, 18 May 2023 03:56:29 +1000 (AEST) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1pzLGG-0003tY-1h; Wed, 17 May 2023 13:48:28 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pzLEt-0001xH-LH for qemu-devel@nongnu.org; Wed, 17 May 2023 13:47:08 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.129.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pzLEr-0004UX-Kj for qemu-devel@nongnu.org; Wed, 17 May 2023 13:47:03 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1684345621; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=fCgR9v3N8PSkB0t2huCh2sJiSmL5LcBYnaM/x43qXsY=; b=DekeseMj0nAV7+R2WhPKYHSBK52BlXyU4z4OKSCJXqSbwswlfd0v/XfimOgQa59gIuIees APnBU/hZcF7UkLQqS/R2aiJSZSDd0QBC4pKRkhrCZaInQGpAfgfU6uSURyj/fIzRD1WCD1 cJ351b9/7xgWqO584ZMgXB8/9FQlny4= Received: from mail-wm1-f72.google.com (mail-wm1-f72.google.com [209.85.128.72]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-175-62tXzSMwNXWjCR7SPsXAAA-1; Wed, 17 May 2023 13:46:59 -0400 X-MC-Unique: 62tXzSMwNXWjCR7SPsXAAA-1 Received: by mail-wm1-f72.google.com with SMTP id 5b1f17b1804b1-3f4fa463282so7693395e9.0 for ; Wed, 17 May 2023 10:46:59 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1684345618; x=1686937618; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=fCgR9v3N8PSkB0t2huCh2sJiSmL5LcBYnaM/x43qXsY=; b=O5bmWZpt7sTez0/GiYXXIrqqKA1Dhc1wLO2ztZK9q/SHNlIbOmPZq4vZIyT5s/fvd/ cGBb2PqlZwa5VXBto61cb/tDWSGfnP/TaxsXDTw2TtXu6MlNvDSbhBCWqvRlrYLSQ8q1 HUXcqZ4QBog6vIId/npkjFvonfycZlW1WmfFZiLQ1xxc97SOmFTph/zCMPlIFYEgS6FX qWk8Opw0K87ilZT/012m4bfenZ47CWLe6036G4RGQqlTtccHI++IJUw7YgPrigoFr1GP N6RmG2kVGl42YVsXymRLFDbU1xiGGkRugjZUKBMqn/gKErTWo+J6V3zeJydjCTgDuGfY 2WdA== X-Gm-Message-State: AC+VfDzdueSV7W+fzKkfER5iiHJTMOMeJMx1+MWXQ4H7g2JULUqGZ54h cYrE5D5vVRad3xmyXdV4e4xIH8Y6AbImN9ghP70XNFRCcWwn/iOzwf2UfpeVqvSxic5LizcYgFT j3IdZMKcXCaPTzGLcLCXVJ0cgechyGOr7sLBz1h6/3uOWpHftDXe5MEzb1WUr5+P8maCpZzVcY7 c= X-Received: by 2002:a1c:f304:0:b0:3f1:979f:a734 with SMTP id q4-20020a1cf304000000b003f1979fa734mr28757585wmq.11.1684345617858; Wed, 17 May 2023 10:46:57 -0700 (PDT) X-Google-Smtp-Source: ACHHUZ6KyGgmSShXGzqJ+BxvOeW+wCOxdVlRURVChF7M3YdPKooND2/sTrRQ6MJMiH59oVVe/PBOGA== X-Received: by 2002:a1c:f304:0:b0:3f1:979f:a734 with SMTP id q4-20020a1cf304000000b003f1979fa734mr28757572wmq.11.1684345617465; Wed, 17 May 2023 10:46:57 -0700 (PDT) Received: from [192.168.10.118] ([2001:b07:6468:f312:9af8:e5f5:7516:fa89]) by smtp.gmail.com with ESMTPSA id u2-20020a05600c00c200b003f4fbd9cdb3sm2851102wmm.34.2023.05.17.10.46.56 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 17 May 2023 10:46:56 -0700 (PDT) From: Paolo Bonzini To: qemu-devel@nongnu.org Cc: =?utf-8?q?Marc-Andr=C3=A9_Lureau?= Subject: [PULL 58/68] build: move stack protector flag selection to meson Date: Wed, 17 May 2023 19:45:10 +0200 Message-Id: <20230517174520.887405-59-pbonzini@redhat.com> X-Mailer: git-send-email 2.40.1 In-Reply-To: <20230517174520.887405-1-pbonzini@redhat.com> References: <20230517174520.887405-1-pbonzini@redhat.com> MIME-Version: 1.0 Received-SPF: pass client-ip=170.10.129.124; envelope-from=pbonzini@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Reviewed-by: Marc-AndrĂ© Lureau Signed-off-by: Paolo Bonzini --- configure | 44 ++--------------------------------- meson.build | 28 +++++++++++++++++++++- meson_options.txt | 2 ++ scripts/meson-buildoptions.sh | 3 +++ 4 files changed, 34 insertions(+), 43 deletions(-) diff --git a/configure b/configure index 470ed9f8c201..44d70803f5ce 100755 --- a/configure +++ b/configure @@ -180,7 +180,7 @@ compile_prog() { local_cflags="$1" local_ldflags="$2" do_cc $CFLAGS $EXTRA_CFLAGS $CONFIGURE_CFLAGS $QEMU_CFLAGS $local_cflags -o $TMPE $TMPC \ - $LDFLAGS $EXTRA_LDFLAGS $CONFIGURE_LDFLAGS $QEMU_LDFLAGS $local_ldflags + $LDFLAGS $EXTRA_LDFLAGS $CONFIGURE_LDFLAGS $local_ldflags } # symbolically link $1 to $2. Portable version of "ln -sf". @@ -226,7 +226,6 @@ static="no" cross_compile="no" cross_prefix="" host_cc="cc" -stack_protector="" use_containers="yes" gdb_bin=$(command -v "gdb-multiarch" || command -v "gdb") gdb_arches="" @@ -394,8 +393,6 @@ sdl2_config="${SDL2_CONFIG-${cross_prefix}sdl2-config}" QEMU_CFLAGS="-fno-strict-aliasing -fno-common -fwrapv" QEMU_CFLAGS="-D_GNU_SOURCE -D_FILE_OFFSET_BITS=64 -D_LARGEFILE_SOURCE $QEMU_CFLAGS" -QEMU_LDFLAGS= - # Flags that are needed during configure but later taken care of by Meson CONFIGURE_CFLAGS="-std=gnu11 -Wall" CONFIGURE_LDFLAGS= @@ -824,10 +821,6 @@ for opt do ;; --disable-werror) werror="no" ;; - --enable-stack-protector) stack_protector="yes" - ;; - --disable-stack-protector) stack_protector="no" - ;; --enable-cfi) cfi="true"; meson_option_add -Db_lto=true @@ -998,7 +991,6 @@ Advanced options (experts only): --with-devices-ARCH=NAME override default configs/devices --enable-debug enable common debug build options --disable-werror disable compilation abort on warning - --disable-stack-protector disable compiler-provided stack protection --cpu=CPU Build for host CPU [$cpu] --enable-plugins enable plugins via shared library loading @@ -1257,7 +1249,7 @@ EOF optflag="$(echo $1 | sed -e 's/^-Wno-/-W/')" do_objc -Werror $optflag \ $OBJCFLAGS $EXTRA_OBJCFLAGS $CONFIGURE_OBJCFLAGS $QEMU_OBJCFLAGS \ - -o $TMPE $TMPM $QEMU_LDFLAGS + -o $TMPE $TMPM } for flag in $gcc_flags; do @@ -1269,37 +1261,6 @@ for flag in $gcc_flags; do fi done -if test "$stack_protector" != "no"; then - cat > $TMPC << EOF -int main(int argc, char *argv[]) -{ - char arr[64], *p = arr, *c = argv[argc - 1]; - while (*c) { - *p++ = *c++; - } - return 0; -} -EOF - gcc_flags="-fstack-protector-strong -fstack-protector-all" - sp_on=0 - for flag in $gcc_flags; do - # We need to check both a compile and a link, since some compiler - # setups fail only on a .c->.o compile and some only at link time - if compile_object "-Werror $flag" && - compile_prog "-Werror $flag" ""; then - QEMU_CFLAGS="$QEMU_CFLAGS $flag" - QEMU_LDFLAGS="$QEMU_LDFLAGS $flag" - sp_on=1 - break - fi - done - if test "$stack_protector" = yes; then - if test $sp_on = 0; then - error_exit "Stack protector not supported" - fi - fi -fi - if test "$static" = "yes" ; then if test "$plugins" = "yes"; then error_exit "static and plugins are mutually incompatible" @@ -2057,7 +2018,6 @@ echo "PKG_CONFIG=${pkg_config}" >> $config_host_mak echo "CC=$cc" >> $config_host_mak echo "QEMU_CFLAGS=$QEMU_CFLAGS" >> $config_host_mak echo "QEMU_OBJCFLAGS=$QEMU_OBJCFLAGS" >> $config_host_mak -echo "QEMU_LDFLAGS=$QEMU_LDFLAGS" >> $config_host_mak echo "EXESUF=$EXESUF" >> $config_host_mak # use included Linux headers diff --git a/meson.build b/meson.build index f410afe8b044..fff488f63f6d 100644 --- a/meson.build +++ b/meson.build @@ -200,7 +200,7 @@ foreach arg : config_host['QEMU_CFLAGS'].split() endif endforeach qemu_objcflags = config_host['QEMU_OBJCFLAGS'].split() -qemu_ldflags = config_host['QEMU_LDFLAGS'].split() +qemu_ldflags = [] if get_option('gprof') qemu_common_flags += ['-p'] @@ -211,6 +211,32 @@ if get_option('prefer_static') qemu_ldflags += get_option('b_pie') ? '-static-pie' : '-static' endif +if not get_option('stack_protector').disabled() + stack_protector_probe = ''' + int main(int argc, char *argv[]) + { + char arr[64], *p = arr, *c = argv[argc - 1]; + while (*c) { + *p++ = *c++; + } + return 0; + }''' + have_stack_protector = false + foreach arg : ['-fstack-protector-strong', '-fstack-protector-all'] + # We need to check both a compile and a link, since some compiler + # setups fail only on a .c->.o compile and some only at link time + if cc.compiles(stack_protector_probe, args: ['-Werror', arg]) and \ + cc.links(stack_protector_probe, args: ['-Werror', arg]) + have_stack_protector = true + qemu_cflags += arg + qemu_ldflags += arg + break + endif + endforeach + get_option('stack_protector') \ + .require(have_stack_protector, error_message: 'Stack protector not supported') +endif + coroutine_backend = get_option('coroutine_backend') ucontext_probe = ''' #include diff --git a/meson_options.txt b/meson_options.txt index 69de30c83db0..43c3ddfd48fc 100644 --- a/meson_options.txt +++ b/meson_options.txt @@ -91,6 +91,8 @@ option('sanitizers', type: 'boolean', value: false, description: 'enable default sanitizers') option('tsan', type: 'boolean', value: false, description: 'enable thread sanitizer') +option('stack_protector', type: 'feature', value: 'auto', + description: 'compiler-provided stack protection') option('cfi', type: 'boolean', value: false, description: 'Control-Flow Integrity (CFI)') option('cfi_debug', type: 'boolean', value: false, diff --git a/scripts/meson-buildoptions.sh b/scripts/meson-buildoptions.sh index 032fcea948d4..28b6e8385bc1 100644 --- a/scripts/meson-buildoptions.sh +++ b/scripts/meson-buildoptions.sh @@ -165,6 +165,7 @@ meson_options_help() { printf "%s\n" ' sparse sparse checker' printf "%s\n" ' spice Spice server support' printf "%s\n" ' spice-protocol Spice protocol support' + printf "%s\n" ' stack-protector compiler-provided stack protection' printf "%s\n" ' tcg TCG support' printf "%s\n" ' tools build support utilities that come with QEMU' printf "%s\n" ' tpm TPM support' @@ -441,6 +442,8 @@ _meson_option_parse() { --disable-spice) printf "%s" -Dspice=disabled ;; --enable-spice-protocol) printf "%s" -Dspice_protocol=enabled ;; --disable-spice-protocol) printf "%s" -Dspice_protocol=disabled ;; + --enable-stack-protector) printf "%s" -Dstack_protector=enabled ;; + --disable-stack-protector) printf "%s" -Dstack_protector=disabled ;; --enable-strip) printf "%s" -Dstrip=true ;; --disable-strip) printf "%s" -Dstrip=false ;; --sysconfdir=*) quote_sh "-Dsysconfdir=$2" ;; From patchwork Wed May 17 17:45:11 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Paolo Bonzini X-Patchwork-Id: 1782878 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=nongnu.org (client-ip=209.51.188.17; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=) Authentication-Results: legolas.ozlabs.org; dkim=pass (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=OOrlytlC; dkim-atps=neutral Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-ECDSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4QM1CP5M5hz20dn for ; Thu, 18 May 2023 04:02:09 +1000 (AEST) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1pzLH4-0005D6-RS; Wed, 17 May 2023 13:49:18 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pzLEw-00022o-I0 for qemu-devel@nongnu.org; Wed, 17 May 2023 13:47:08 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pzLEt-0004Uw-49 for qemu-devel@nongnu.org; Wed, 17 May 2023 13:47:04 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1684345622; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=nwLn6aNVqQkCPu/7+/R56Ne8zlCyQE9FIpZx6bYWA2Q=; b=OOrlytlCFEtvQfr+R54r3M69hfE0LYQe+ZQ0YknPe1tLLmtPWgXif/z4xOUS8XXRCjGKic pTtiExZWchhgDJXMlOemCnQJ/joHPpBTIYXgQO9rvtQtILbzBb/Not2lrZH0qGEIuIjvFC tshkh2lych6dnlCZ1Xw3yTTPzKnGk7c= Received: from mail-wm1-f69.google.com (mail-wm1-f69.google.com [209.85.128.69]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-620-Xg236eKRMk-QV9wXCMqxYA-1; Wed, 17 May 2023 13:47:01 -0400 X-MC-Unique: Xg236eKRMk-QV9wXCMqxYA-1 Received: by mail-wm1-f69.google.com with SMTP id 5b1f17b1804b1-3f420742d40so4453785e9.2 for ; Wed, 17 May 2023 10:47:01 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1684345619; x=1686937619; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=nwLn6aNVqQkCPu/7+/R56Ne8zlCyQE9FIpZx6bYWA2Q=; b=ZioRqb2Jr2Up4Eq+OrzJJX4B8fBEQlbklGgs+ksWAaG/8IHGWbXtMfaXeLCuy9ij3q KVhcHWKqDki5Q4vapbGbyDmv1JvwU++U4ziLDSrQFalpTtbdtZ+ZjeEs7m7TgYaS2RnH jSsbP4BHNzpqt02lRWo+o5Waid5W3QbY7/Phi8YGifEZKFTV0Mo0bFjdfz8FE98/Ha6d n5pymMpeNX3GfP/maJNMHgmRHR90RWmS9OOggKMHZHkZZFrPm6hGyBKTouHhW7UsIgIq nGvB+lcRmbav2zbwI7Z1paKfbF5wRyq0SGXa0YE9pwFNyN6iuv/nmVbQZ21L1DphIw4w uiyw== X-Gm-Message-State: AC+VfDyKzJzpXggcd45U321vBtZBoddam6IMyKKWuGIXHYUAu1THX2BQ YmFuXPW+CJRUOslPr3ufXXlA8WhKhhc5FxEVObsCVPJU4Nes3IbpLLDfRNnfeW7bADLvGNifZHx UkRjIMbfSwGofUlqPiLey64ME6EjwxdWzhNCMoCrAVD0xJCuZ2hSNHjwqAIWYgei/Xr4EL/0/Pb c= X-Received: by 2002:a05:600c:21c8:b0:3f4:e9f6:1c3b with SMTP id x8-20020a05600c21c800b003f4e9f61c3bmr16208584wmj.12.1684345619428; Wed, 17 May 2023 10:46:59 -0700 (PDT) X-Google-Smtp-Source: ACHHUZ7lxyyD5JHpFo1I1vC3P7f1GNOkg8D8MI5B9ePEhtSy1VmnJJzSzXyQ+am4v989lohQ4M5c7w== X-Received: by 2002:a05:600c:21c8:b0:3f4:e9f6:1c3b with SMTP id x8-20020a05600c21c800b003f4e9f61c3bmr16208571wmj.12.1684345619033; Wed, 17 May 2023 10:46:59 -0700 (PDT) Received: from [192.168.10.118] ([2001:b07:6468:f312:9af8:e5f5:7516:fa89]) by smtp.gmail.com with ESMTPSA id m6-20020a7bca46000000b003f4bef65a65sm2860584wml.28.2023.05.17.10.46.57 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 17 May 2023 10:46:58 -0700 (PDT) From: Paolo Bonzini To: qemu-devel@nongnu.org Cc: =?utf-8?q?Marc-Andr=C3=A9_Lureau?= Subject: [PULL 59/68] build: move warning flag selection to meson Date: Wed, 17 May 2023 19:45:11 +0200 Message-Id: <20230517174520.887405-60-pbonzini@redhat.com> X-Mailer: git-send-email 2.40.1 In-Reply-To: <20230517174520.887405-1-pbonzini@redhat.com> References: <20230517174520.887405-1-pbonzini@redhat.com> MIME-Version: 1.0 Received-SPF: pass client-ip=170.10.133.124; envelope-from=pbonzini@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Meson already knows to test with the positive form of the flag, which simplifies the test. Warnings are now tested explicitly for the C++ compiler, instead of hardcoding those that are only available for the C language. At this point all compiler flags in QEMU_CFLAGS are global and only depend on the OS. No feature tests are performed in configure. Reviewed-by: Marc-AndrĂ© Lureau Signed-off-by: Paolo Bonzini --- configure | 86 ---------------------------------------- contrib/plugins/Makefile | 3 +- meson.build | 70 ++++++++++++++++++++++---------- 3 files changed, 49 insertions(+), 110 deletions(-) diff --git a/configure b/configure index 44d70803f5ce..eed1aec31c23 100755 --- a/configure +++ b/configure @@ -79,7 +79,6 @@ fi TMPB="qemu-conf" TMPC="${TMPDIR1}/${TMPB}.c" TMPO="${TMPDIR1}/${TMPB}.o" -TMPM="${TMPDIR1}/${TMPB}.m" TMPE="${TMPDIR1}/${TMPB}.exe" rm -f config.log @@ -162,15 +161,6 @@ do_cc() { do_compiler_werror "$cc" $CPU_CFLAGS "$@" } -do_objc() { - do_compiler_werror "$objcc" $CPU_CFLAGS "$@" -} - -# Append $2 to the variable named $1, with space separation -add_to() { - eval $1=\${$1:+\"\$$1 \"}\$2 -} - compile_object() { local_cflags="$1" do_cc $CFLAGS $EXTRA_CFLAGS $CONFIGURE_CFLAGS $QEMU_CFLAGS $local_cflags -c -o $TMPO $TMPC @@ -1186,81 +1176,6 @@ if ! compile_prog "" "" ; then error_exit "You need at least GCC v7.4 or Clang v10.0 (or XCode Clang v12.0)" fi -# Accumulate -Wfoo and -Wno-bar separately. -# We will list all of the enable flags first, and the disable flags second. -# Note that we do not add -Werror, because that would enable it for all -# configure tests. If a configure test failed due to -Werror this would -# just silently disable some features, so it's too error prone. - -warn_flags= -add_to warn_flags -Wundef -add_to warn_flags -Wwrite-strings -add_to warn_flags -Wmissing-prototypes -add_to warn_flags -Wstrict-prototypes -add_to warn_flags -Wredundant-decls -add_to warn_flags -Wold-style-declaration -add_to warn_flags -Wold-style-definition -add_to warn_flags -Wtype-limits -add_to warn_flags -Wformat-security -add_to warn_flags -Wformat-y2k -add_to warn_flags -Winit-self -add_to warn_flags -Wignored-qualifiers -add_to warn_flags -Wempty-body -add_to warn_flags -Wnested-externs -add_to warn_flags -Wendif-labels -add_to warn_flags -Wexpansion-to-defined -add_to warn_flags -Wimplicit-fallthrough=2 -add_to warn_flags -Wmissing-format-attribute - -if test "$targetos" != "darwin"; then - add_to warn_flags -Wthread-safety -fi - -nowarn_flags= -add_to nowarn_flags -Wno-initializer-overrides -add_to nowarn_flags -Wno-missing-include-dirs -add_to nowarn_flags -Wno-shift-negative-value -add_to nowarn_flags -Wno-string-plus-int -add_to nowarn_flags -Wno-typedef-redefinition -add_to nowarn_flags -Wno-tautological-type-limit-compare -add_to nowarn_flags -Wno-psabi -add_to nowarn_flags -Wno-gnu-variable-sized-type-not-at-end - -gcc_flags="$warn_flags $nowarn_flags" - -cc_has_warning_flag() { - write_c_skeleton; - - # Use the positive sense of the flag when testing for -Wno-wombat - # support (gcc will happily accept the -Wno- form of unknown - # warning options). - optflag="$(echo $1 | sed -e 's/^-Wno-/-W/')" - compile_prog "-Werror $optflag" "" -} - -objcc_has_warning_flag() { - cat > $TMPM <> $config_host_mak echo "PKG_CONFIG=${pkg_config}" >> $config_host_mak echo "CC=$cc" >> $config_host_mak echo "QEMU_CFLAGS=$QEMU_CFLAGS" >> $config_host_mak -echo "QEMU_OBJCFLAGS=$QEMU_OBJCFLAGS" >> $config_host_mak echo "EXESUF=$EXESUF" >> $config_host_mak # use included Linux headers diff --git a/contrib/plugins/Makefile b/contrib/plugins/Makefile index 8a316cd76f2f..b2b9db9f51af 100644 --- a/contrib/plugins/Makefile +++ b/contrib/plugins/Makefile @@ -27,8 +27,7 @@ SONAMES := $(addsuffix .so,$(addprefix lib,$(NAMES))) # The main QEMU uses Glib extensively so it's perfectly fine to use it # in plugins (which many example do). CFLAGS := $(shell $(PKG_CONFIG) --cflags glib-2.0) -CFLAGS += -fPIC -Wall $(filter -W%, $(QEMU_CFLAGS)) -CFLAGS += $(if $(findstring no-psabi,$(QEMU_CFLAGS)),-Wpsabi) +CFLAGS += -fPIC -Wall CFLAGS += $(if $(CONFIG_DEBUG_TCG), -ggdb -O0) CFLAGS += -I$(SRC_PATH)/include/qemu diff --git a/meson.build b/meson.build index fff488f63f6d..e7a7a1624afa 100644 --- a/meson.build +++ b/meson.build @@ -190,16 +190,8 @@ endif # Compiler flags # ################## -qemu_common_flags = [] +qemu_common_flags = config_host['QEMU_CFLAGS'].split() qemu_cflags = [] -foreach arg : config_host['QEMU_CFLAGS'].split() - if arg.startswith('-W') - qemu_cflags += arg - else - qemu_common_flags += arg - endif -endforeach -qemu_objcflags = config_host['QEMU_OBJCFLAGS'].split() qemu_ldflags = [] if get_option('gprof') @@ -369,20 +361,47 @@ endif add_global_arguments(qemu_common_flags, native: false, language: all_languages) add_global_link_arguments(qemu_ldflags, native: false, language: all_languages) +# Collect warnings that we want to enable + +warn_flags = [ + '-Wundef', + '-Wwrite-strings', + '-Wmissing-prototypes', + '-Wstrict-prototypes', + '-Wredundant-decls', + '-Wold-style-declaration', + '-Wold-style-definition', + '-Wtype-limits', + '-Wformat-security', + '-Wformat-y2k', + '-Winit-self', + '-Wignored-qualifiers', + '-Wempty-body', + '-Wnested-externs', + '-Wendif-labels', + '-Wexpansion-to-defined', + '-Wimplicit-fallthrough=2', + '-Wmissing-format-attribute', + '-Wno-initializer-overrides', + '-Wno-missing-include-dirs', + '-Wno-shift-negative-value', + '-Wno-string-plus-int', + '-Wno-typedef-redefinition', + '-Wno-tautological-type-limit-compare', + '-Wno-psabi', + '-Wno-gnu-variable-sized-type-not-at-end', +] + +if targetos != 'darwin' + warn_flags += ['-Wthread-safety'] +endif + # Check that the C++ compiler exists and works with the C compiler. link_language = 'c' linker = cc qemu_cxxflags = [] if 'cpp' in all_languages - add_global_arguments(['-D__STDC_LIMIT_MACROS', '-D__STDC_CONSTANT_MACROS', '-D__STDC_FORMAT_MACROS'], - native: false, language: 'cpp') - foreach k: qemu_cflags - if k not in ['-Wstrict-prototypes', '-Wmissing-prototypes', '-Wnested-externs', - '-Wold-style-declaration', '-Wold-style-definition', '-Wredundant-decls'] - qemu_cxxflags += [k] - endif - endforeach - + qemu_cxxflags = ['-D__STDC_LIMIT_MACROS', '-D__STDC_CONSTANT_MACROS', '-D__STDC_FORMAT_MACROS'] + qemu_cflags if cxx.links(files('scripts/main.c'), args: qemu_cflags) link_language = 'cpp' linker = cxx @@ -402,9 +421,16 @@ if get_option('optimization') != '0' and targetos == 'linux' endif endif -add_project_arguments(qemu_cflags, native: false, language: 'c') -add_project_arguments(qemu_cxxflags, native: false, language: 'cpp') -add_project_arguments(qemu_objcflags, native: false, language: 'objc') +add_project_arguments(qemu_cflags, native: false, language: 'objc') +add_project_arguments(cc.get_supported_arguments(warn_flags), native: false, language: 'c') +if 'cpp' in all_languages + add_project_arguments(qemu_cxxflags, native: false, language: 'cpp') + add_project_arguments(cxx.get_supported_arguments(warn_flags), native: false, language: 'cpp') +endif +if 'objc' in all_languages + # Note sanitizer flags are not applied to Objective-C sources! + add_project_arguments(objc.get_supported_arguments(warn_flags), native: false, language: 'objc') +endif if targetos == 'linux' add_project_arguments('-isystem', meson.current_source_dir() / 'linux-headers', '-isystem', 'linux-headers', @@ -3986,7 +4012,7 @@ if 'cpp' in all_languages summary_info += {'QEMU_CXXFLAGS': ' '.join(qemu_common_flags + qemu_cxxflags)} endif if 'objc' in all_languages - summary_info += {'QEMU_OBJCFLAGS': ' '.join(qemu_common_flags + qemu_objcflags)} + summary_info += {'QEMU_OBJCFLAGS': ' '.join(qemu_common_flags)} endif summary_info += {'QEMU_LDFLAGS': ' '.join(qemu_ldflags)} summary_info += {'profiler': get_option('profiler')} From patchwork Wed May 17 17:45:12 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Paolo Bonzini X-Patchwork-Id: 1782872 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=nongnu.org (client-ip=209.51.188.17; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=) Authentication-Results: legolas.ozlabs.org; dkim=pass (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=dK1SLF4c; dkim-atps=neutral Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-ECDSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4QM16Z0Hqvz20dq for ; Thu, 18 May 2023 03:57:57 +1000 (AEST) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1pzLH6-0005Ps-L6; Wed, 17 May 2023 13:49:20 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pzLF0-00025A-Au for qemu-devel@nongnu.org; Wed, 17 May 2023 13:47:10 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.129.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pzLEx-0004X5-Cp for qemu-devel@nongnu.org; Wed, 17 May 2023 13:47:10 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1684345626; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=jig9o1Y8Sz+QnnoZxr0j8M49vPjfPTSpyMINpp/zs4w=; b=dK1SLF4cinA6pMOybgsQEKAo7Zw6Q1e7L8Wf6q25M2LAIC9GhTFTOKNysHb/sjShQvoJM3 +Ef0Cwb7vJjMBqXUq6sCosCtmT7yX8uu36+bPY4jlUjcJcr0RC1NCttlsf4W0snDsVEpzc LzyYejiPCXgeqPG7gAn6J2DsES8AgAs= Received: from mail-wr1-f71.google.com (mail-wr1-f71.google.com [209.85.221.71]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-133-tcfj2NH7M3mSL5q0QddmFw-1; Wed, 17 May 2023 13:47:02 -0400 X-MC-Unique: tcfj2NH7M3mSL5q0QddmFw-1 Received: by mail-wr1-f71.google.com with SMTP id ffacd0b85a97d-30640be4fd0so449680f8f.1 for ; Wed, 17 May 2023 10:47:02 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1684345621; x=1686937621; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=jig9o1Y8Sz+QnnoZxr0j8M49vPjfPTSpyMINpp/zs4w=; b=Sn5tRDqQpLqABiwfhLl71jfb1Qmp29So+WYai0SlSTU1mkE2iudeUNDRNpbKSR/qHB Bjb2u1djC4/lnjBQTjoOyWVud/Iubq50I8miQLWvj/AqTnAsc7huHQ100fbUtNuxJ9Zh mF1miVPy028SG8JilY9vgakSimtXZ6tXAiCURqIiLcUBdr3K5F6t5Lv10p4Txlb+XCcp 6Ajbh8y4PFTPVZYnUbFumIgcA0oIUnsve6E44MsAdWkeCHXYfWchTNFxMOgK+hYgw3Rc zIPSkN31pemNUPlvO1oFkkAgCqy3g4uefbE455TXW3FdscGkaEn8tCec+TEoBfCz9GuL jqDQ== X-Gm-Message-State: AC+VfDyezzGJVFmbJsYPBtITa9rNhVkJNcLHyIbYGfO1yAZ9MSdaDZgo lsAls8TCegKtbsD3Og/CJ2Drq0y9ATm4CPU1Q97PYblYwd4MWOjBxhhi0JQTa/i8YQ0HJ2yxW5r WcszmYrkvyv0Oo1bvm07mDxKkXkgAS3Dr+JvonLFLRRFLQtoCBB15F3L01MOpvLx1iBE3sMvqod 4= X-Received: by 2002:adf:fd06:0:b0:309:41f7:e40e with SMTP id e6-20020adffd06000000b0030941f7e40emr1208247wrr.28.1684345620971; Wed, 17 May 2023 10:47:00 -0700 (PDT) X-Google-Smtp-Source: ACHHUZ5XQB7kjPejZRpSQXAMn1UHfOqpw7Z9jT+5tCcQ3KFvBCN27C/4wUbyOyvGURSTkAGxK4w8ZQ== X-Received: by 2002:adf:fd06:0:b0:309:41f7:e40e with SMTP id e6-20020adffd06000000b0030941f7e40emr1208236wrr.28.1684345620578; Wed, 17 May 2023 10:47:00 -0700 (PDT) Received: from [192.168.10.118] ([2001:b07:6468:f312:9af8:e5f5:7516:fa89]) by smtp.gmail.com with ESMTPSA id n17-20020a5d6611000000b003079c402762sm3527532wru.19.2023.05.17.10.46.59 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 17 May 2023 10:46:59 -0700 (PDT) From: Paolo Bonzini To: qemu-devel@nongnu.org Cc: =?utf-8?q?Marc-Andr=C3=A9_Lureau?= Subject: [PULL 60/68] build: move remaining compiler flag tests to meson Date: Wed, 17 May 2023 19:45:12 +0200 Message-Id: <20230517174520.887405-61-pbonzini@redhat.com> X-Mailer: git-send-email 2.40.1 In-Reply-To: <20230517174520.887405-1-pbonzini@redhat.com> References: <20230517174520.887405-1-pbonzini@redhat.com> MIME-Version: 1.0 Received-SPF: pass client-ip=170.10.129.124; envelope-from=pbonzini@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Remove the only remaining uses of QEMU_CFLAGS. Now that no feature tests are done in configure, it is possible to remove CONFIGURE_CFLAGS and CONFIGURE_LDFLAGS as well. Reviewed-by: Marc-AndrĂ© Lureau Signed-off-by: Paolo Bonzini --- configure | 79 ++++++++--------------------------------------------- meson.build | 49 ++++++++++++++++++++++++++++++++- 2 files changed, 59 insertions(+), 69 deletions(-) diff --git a/configure b/configure index eed1aec31c23..6e78f6199feb 100755 --- a/configure +++ b/configure @@ -163,14 +163,14 @@ do_cc() { compile_object() { local_cflags="$1" - do_cc $CFLAGS $EXTRA_CFLAGS $CONFIGURE_CFLAGS $QEMU_CFLAGS $local_cflags -c -o $TMPO $TMPC + do_cc $CFLAGS $EXTRA_CFLAGS $local_cflags -c -o $TMPO $TMPC } compile_prog() { local_cflags="$1" local_ldflags="$2" - do_cc $CFLAGS $EXTRA_CFLAGS $CONFIGURE_CFLAGS $QEMU_CFLAGS $local_cflags -o $TMPE $TMPC \ - $LDFLAGS $EXTRA_LDFLAGS $CONFIGURE_LDFLAGS $local_ldflags + do_cc $CFLAGS $EXTRA_CFLAGS $local_cflags -o $TMPE $TMPC \ + $LDFLAGS $EXTRA_LDFLAGS $local_ldflags } # symbolically link $1 to $2. Portable version of "ln -sf". @@ -375,19 +375,6 @@ windmc="${WINDMC-${cross_prefix}windmc}" pkg_config="${PKG_CONFIG-${cross_prefix}pkg-config}" sdl2_config="${SDL2_CONFIG-${cross_prefix}sdl2-config}" -# default flags for all hosts -# We use -fwrapv to tell the compiler that we require a C dialect where -# left shift of signed integers is well defined and has the expected -# 2s-complement style results. (Both clang and gcc agree that it -# provides these semantics.) -QEMU_CFLAGS="-fno-strict-aliasing -fno-common -fwrapv" -QEMU_CFLAGS="-D_GNU_SOURCE -D_FILE_OFFSET_BITS=64 -D_LARGEFILE_SOURCE $QEMU_CFLAGS" - -# Flags that are needed during configure but later taken care of by Meson -CONFIGURE_CFLAGS="-std=gnu11 -Wall" -CONFIGURE_LDFLAGS= - - check_define() { cat > $TMPC < $TMPC << EOF -static int sfaa(int *ptr) -{ - return __sync_fetch_and_and(ptr, 0); -} - -int main(void) -{ - int val = 42; - val = __sync_val_compare_and_swap(&val, 0, 1); - sfaa(&val); - return val; -} -EOF - if ! compile_prog "" "" ; then - QEMU_CFLAGS="-march=i486 $QEMU_CFLAGS" - fi -fi if test -z "${target_list+xxx}" ; then default_targets=yes @@ -1931,7 +1875,6 @@ echo "MESON=$meson" >> $config_host_mak echo "NINJA=$ninja" >> $config_host_mak echo "PKG_CONFIG=${pkg_config}" >> $config_host_mak echo "CC=$cc" >> $config_host_mak -echo "QEMU_CFLAGS=$QEMU_CFLAGS" >> $config_host_mak echo "EXESUF=$EXESUF" >> $config_host_mak # use included Linux headers diff --git a/meson.build b/meson.build index e7a7a1624afa..6098426e7198 100644 --- a/meson.build +++ b/meson.build @@ -190,10 +190,50 @@ endif # Compiler flags # ################## -qemu_common_flags = config_host['QEMU_CFLAGS'].split() +# default flags for all hosts +# We use -fwrapv to tell the compiler that we require a C dialect where +# left shift of signed integers is well defined and has the expected +# 2s-complement style results. (Both clang and gcc agree that it +# provides these semantics.) + +qemu_common_flags = [ + '-D_GNU_SOURCE', '-D_FILE_OFFSET_BITS=64', '-D_LARGEFILE_SOURCE', + '-fno-strict-aliasing', '-fno-common', '-fwrapv' ] qemu_cflags = [] qemu_ldflags = [] +if targetos == 'darwin' + # Disable attempts to use ObjectiveC features in os/object.h since they + # won't work when we're compiling with gcc as a C compiler. + qemu_common_flags += '-DOS_OBJECT_USE_OBJC=0' +elif targetos == 'solaris' + # needed for CMSG_ macros in sys/socket.h + qemu_common_flags += '-D_XOPEN_SOURCE=600' + # needed for TIOCWIN* defines in termios.h + qemu_common_flags += '-D__EXTENSIONS__' +elif targetos == 'haiku' + qemu_common_flags += ['-DB_USE_POSITIVE_POSIX_ERRORS', '-D_BSD_SOURCE', '-fPIC'] +endif + +# __sync_fetch_and_and requires at least -march=i486. Many toolchains +# use i686 as default anyway, but for those that don't, an explicit +# specification is necessary +if host_arch == 'i386' and not cc.links(''' + static int sfaa(int *ptr) + { + return __sync_fetch_and_and(ptr, 0); + } + + int main(void) + { + int val = 42; + val = __sync_val_compare_and_swap(&val, 0, 1); + sfaa(&val); + return val; + }''') + qemu_common_flags = ['-march=i486'] + qemu_common_flags +endif + if get_option('gprof') qemu_common_flags += ['-p'] qemu_ldflags += ['-p'] @@ -203,6 +243,13 @@ if get_option('prefer_static') qemu_ldflags += get_option('b_pie') ? '-static-pie' : '-static' endif +# Meson currently only handles pie as a boolean for now, so if the user +# has explicitly disabled PIE we need to extend our cflags. +if not get_option('b_pie') + qemu_cflags += cc.get_supported_arguments('-fno-pie') + qemu_ldflags += cc.get_supported_link_arguments('-no-pie') +endif + if not get_option('stack_protector').disabled() stack_protector_probe = ''' int main(int argc, char *argv[]) From patchwork Wed May 17 17:45:13 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Paolo Bonzini X-Patchwork-Id: 1782870 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=nongnu.org (client-ip=209.51.188.17; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=) Authentication-Results: legolas.ozlabs.org; dkim=pass (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=i8nzUYIe; dkim-atps=neutral Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-ECDSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4QM16Z0N5nz20dr for ; Thu, 18 May 2023 03:57:57 +1000 (AEST) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1pzLH6-0005R7-VF; Wed, 17 May 2023 13:49:21 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pzLEz-000241-BO for qemu-devel@nongnu.org; Wed, 17 May 2023 13:47:10 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.129.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pzLEw-0004Wk-NX for qemu-devel@nongnu.org; Wed, 17 May 2023 13:47:08 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1684345626; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=WFIDugn5duTxyfakELQCByov4jsyEnq4Yu1Y8iJ0ISI=; b=i8nzUYIeR8jwfpq7NJFJH0iZaJIx9gHyd+8VdqW+ag29hDDcByhVzw4yZQ+gZ8fyngyCAX Iw9NTFCdlkBEKvFcVlcLoY6Y+eic18/QzxL8yRBC9jN42R+U7E3cIyD4MiMZAVI5CwrceJ 6GAcfum4yDLuEme69REfUKCMT+gUiBw= Received: from mail-wr1-f71.google.com (mail-wr1-f71.google.com [209.85.221.71]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-373-PQDKK7NTNWipgk6v6JQqKg-1; Wed, 17 May 2023 13:47:04 -0400 X-MC-Unique: PQDKK7NTNWipgk6v6JQqKg-1 Received: by mail-wr1-f71.google.com with SMTP id ffacd0b85a97d-306311a2b99so469615f8f.1 for ; Wed, 17 May 2023 10:47:03 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1684345622; x=1686937622; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=WFIDugn5duTxyfakELQCByov4jsyEnq4Yu1Y8iJ0ISI=; b=F71hTueehdtFjAfiq1D8dkuLSL7GFZ7ElA+l+KgoDFdCJ22kaPNf+zZYgoWnATi2hD WyqLSlUbaCDZihtiCy0njtj6COXfUH4Y+xF3SB+L+MgUPfTE8+meMULs8Fmd8yHg4dbm 4uczYFG5b+vGtCGbmo0ePi6mAvkU0JVF3R2z77c5cfkyrcSGkNGTnbAUQunCZJIw0Gg6 GTJZ56EhP1wODi3dRb1kaXkLSNsE4EXqmzrRfK3phoAmfrBMQlLrWLBxFtMP9g225l86 B90A4hatDmju31QXc5GG/1DuyUGandvlhdUt/Hh+O0NwIBvXlBs2xiSgjFCHGsR4n+kZ l9AA== X-Gm-Message-State: AC+VfDy5Ex6Nv3o2wTTSVAUAUJlnAzsUyefmLzRfXWhlhQnLTbYqA7DP lFvriF/D8nVGhsToafGqMN3KrZJc3ValOVIEEku7UsCTPemMx4mcMWYWzck7UPkjHm+x6eVH2tv ASa/gYJygpAl0b23ghbXq8Rh2RpVtygTwX8N5jBsYP2SOmMYi7an5ZU3BT8zT5rcvETsDmYtdND g= X-Received: by 2002:a5d:618e:0:b0:307:9592:846d with SMTP id j14-20020a5d618e000000b003079592846dmr1428575wru.14.1684345622647; Wed, 17 May 2023 10:47:02 -0700 (PDT) X-Google-Smtp-Source: ACHHUZ6y0eQHEQhCoi+sudkUS5YygRCEWvXB5l3t98iA+MpqaLbBeP9IJX1+4BMuOzUK9cGDAHKqCw== X-Received: by 2002:a5d:618e:0:b0:307:9592:846d with SMTP id j14-20020a5d618e000000b003079592846dmr1428561wru.14.1684345622290; Wed, 17 May 2023 10:47:02 -0700 (PDT) Received: from [192.168.10.118] ([2001:b07:6468:f312:9af8:e5f5:7516:fa89]) by smtp.gmail.com with ESMTPSA id g12-20020a7bc4cc000000b003f4fe09aa43sm2981745wmk.8.2023.05.17.10.47.01 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 17 May 2023 10:47:01 -0700 (PDT) From: Paolo Bonzini To: qemu-devel@nongnu.org Subject: [PULL 61/68] build: move compiler version check to meson Date: Wed, 17 May 2023 19:45:13 +0200 Message-Id: <20230517174520.887405-62-pbonzini@redhat.com> X-Mailer: git-send-email 2.40.1 In-Reply-To: <20230517174520.887405-1-pbonzini@redhat.com> References: <20230517174520.887405-1-pbonzini@redhat.com> MIME-Version: 1.0 Received-SPF: pass client-ip=170.10.129.124; envelope-from=pbonzini@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Use the slighly nicer .version_compare() function for GCC; for Clang that is not possible due to the mess that Apple does with version numbers. Signed-off-by: Paolo Bonzini --- configure | 25 ------------------------- meson.build | 20 ++++++++++++++++++++ 2 files changed, 20 insertions(+), 25 deletions(-) diff --git a/configure b/configure index 6e78f6199feb..0dc0395a848f 100755 --- a/configure +++ b/configure @@ -1128,31 +1128,6 @@ if test "$targetos" = "bogus"; then error_exit "Unrecognized host OS (uname -s reports '$(uname -s)')" fi -# Check whether the compiler matches our minimum requirements: -cat > $TMPC << EOF -#if defined(__clang_major__) && defined(__clang_minor__) -# ifdef __apple_build_version__ -# if __clang_major__ < 12 || (__clang_major__ == 12 && __clang_minor__ < 0) -# error You need at least XCode Clang v12.0 to compile QEMU -# endif -# else -# if __clang_major__ < 10 || (__clang_major__ == 10 && __clang_minor__ < 0) -# error You need at least Clang v10.0 to compile QEMU -# endif -# endif -#elif defined(__GNUC__) && defined(__GNUC_MINOR__) -# if __GNUC__ < 7 || (__GNUC__ == 7 && __GNUC_MINOR__ < 4) -# error You need at least GCC v7.4.0 to compile QEMU -# endif -#else -# error You either need GCC or Clang to compiler QEMU -#endif -int main (void) { return 0; } -EOF -if ! compile_prog "" "" ; then - error_exit "You need at least GCC v7.4 or Clang v10.0 (or XCode Clang v12.0)" -fi - if test "$static" = "yes" ; then if test "$plugins" = "yes"; then error_exit "static and plugins are mutually incompatible" diff --git a/meson.build b/meson.build index 6098426e7198..c775f24b79b0 100644 --- a/meson.build +++ b/meson.build @@ -190,6 +190,26 @@ endif # Compiler flags # ################## +foreach lang : all_languages + compiler = meson.get_compiler(lang) + if compiler.get_id() == 'gcc' and compiler.version().version_compare('>=7.4') + # ok + elif compiler.get_id() == 'clang' and compiler.compiles(''' + #ifdef __apple_build_version__ + # if __clang_major__ < 12 || (__clang_major__ == 12 && __clang_minor__ < 0) + # error You need at least XCode Clang v12.0 to compile QEMU + # endif + #else + # if __clang_major__ < 10 || (__clang_major__ == 10 && __clang_minor__ < 0) + # error You need at least Clang v10.0 to compile QEMU + # endif + #endif''') + # ok + else + error('You either need GCC v7.4 or Clang v10.0 (or XCode Clang v12.0) to compile QEMU') + endif +endforeach + # default flags for all hosts # We use -fwrapv to tell the compiler that we require a C dialect where # left shift of signed integers is well defined and has the expected From patchwork Wed May 17 17:45:14 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Paolo Bonzini X-Patchwork-Id: 1782862 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=nongnu.org (client-ip=209.51.188.17; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=) Authentication-Results: legolas.ozlabs.org; dkim=pass (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=HM6sxkrM; dkim-atps=neutral Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-ECDSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4QM14V60wbz20dX for ; Thu, 18 May 2023 03:56:10 +1000 (AEST) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1pzLHB-0005dw-Mp; Wed, 17 May 2023 13:49:25 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pzLFB-0002oz-EZ for qemu-devel@nongnu.org; Wed, 17 May 2023 13:47:24 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.129.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pzLEx-0004XV-Ph for qemu-devel@nongnu.org; Wed, 17 May 2023 13:47:21 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1684345627; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=YcHvFX9e7TrY62C9Kq986qOxDclbTbjX+dadAAdMczg=; b=HM6sxkrM6nGmqIWTDfjZeQ4grfIiLffKI0TTQsmrls8rS9Qq8mWulcwKMhIMze0taBliZl SQ0qN0Q6HppOKgDf8r+92L6QgjEOpYlP4lFRiiOJz3RNY6epMmz6CqgrKc8LbZnGWXU5zs JeFYoNuKZPtEki8geDb/KECBTBZoTyM= Received: from mail-wm1-f70.google.com (mail-wm1-f70.google.com [209.85.128.70]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-479-z5dybUTkO2-K5Q_SWzqc0w-1; Wed, 17 May 2023 13:47:05 -0400 X-MC-Unique: z5dybUTkO2-K5Q_SWzqc0w-1 Received: by mail-wm1-f70.google.com with SMTP id 5b1f17b1804b1-3f33f8ffa95so4720915e9.3 for ; Wed, 17 May 2023 10:47:05 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1684345624; x=1686937624; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=YcHvFX9e7TrY62C9Kq986qOxDclbTbjX+dadAAdMczg=; b=Y0bShyE7ZVu1cOOC3e60iC60ZDtx8WglMsFfSY5SzciKs33DCLDYg0L/wEafzdJxqp qNVn+SrCSay/jO0LjrdQbn+qzKV+WghiUwK2B6qBK7vWEqnleKt+qG8QUXHGHMQQFTFh IbpWD5snw/K2CwiOU2vqx9/g+UvPF9wd8vvUnn6KVlPWX8o+j/Gz+CTh2ZmaLkegoz31 YrXYYsL/ZXcWm87ProPQIqcfFJmoJNjgGoV3xJ8vaBU2qXiRq8Y8wB29fVL1pK0yCiXX WtFHeOOdtC2VbdiNanQXXBLrby7vXV2/9CtBz2atC6A8uF/nHa8ZM2ktzEermG3PVkpt 8X2A== X-Gm-Message-State: AC+VfDy2qmRGp2hVtoSPP3IFPWxbQ2BjgxIKIG32jeKPsvyebriscmzQ dlomcsA6kE2lj49s3P76JT5xiH5lZRYaAVGqdaGkr4+OVDz4EVRQ4YO7gURhJicawumPp8v+Rr6 xOUVBcvFJYALLKYC1CJqkH28KfjAr7AZdxea5dSJiL044lp7WsBPF+vQc1nTVBl4rhYX2x5I/iS Y= X-Received: by 2002:a7b:ce95:0:b0:3f4:e853:6a1 with SMTP id q21-20020a7bce95000000b003f4e85306a1mr15122352wmj.38.1684345623918; Wed, 17 May 2023 10:47:03 -0700 (PDT) X-Google-Smtp-Source: ACHHUZ7bys51F56a7U9Q/kcirC1FfsSPjMpH5excGxZUG4oRXkY2SFsHuTleUVKeEpYW2GwTb18+JA== X-Received: by 2002:a7b:ce95:0:b0:3f4:e853:6a1 with SMTP id q21-20020a7bce95000000b003f4e85306a1mr15122340wmj.38.1684345623533; Wed, 17 May 2023 10:47:03 -0700 (PDT) Received: from [192.168.10.118] ([2001:b07:6468:f312:9af8:e5f5:7516:fa89]) by smtp.gmail.com with ESMTPSA id a3-20020a1cf003000000b003f4f9274d10sm2929813wmb.16.2023.05.17.10.47.02 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 17 May 2023 10:47:02 -0700 (PDT) From: Paolo Bonzini To: qemu-devel@nongnu.org Subject: [PULL 62/68] build: move --disable-debug-info to meson Date: Wed, 17 May 2023 19:45:14 +0200 Message-Id: <20230517174520.887405-63-pbonzini@redhat.com> X-Mailer: git-send-email 2.40.1 In-Reply-To: <20230517174520.887405-1-pbonzini@redhat.com> References: <20230517174520.887405-1-pbonzini@redhat.com> MIME-Version: 1.0 Received-SPF: pass client-ip=170.10.129.124; envelope-from=pbonzini@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Signed-off-by: Paolo Bonzini --- configure | 5 ----- scripts/meson-buildoptions.py | 2 ++ scripts/meson-buildoptions.sh | 3 +++ 3 files changed, 5 insertions(+), 5 deletions(-) diff --git a/configure b/configure index 0dc0395a848f..b2172b7b938e 100755 --- a/configure +++ b/configure @@ -693,10 +693,6 @@ for opt do ;; --cross-prefix-*) ;; - --enable-debug-info) meson_option_add -Ddebug=true - ;; - --disable-debug-info) meson_option_add -Ddebug=false - ;; --enable-docs) docs=enabled ;; --disable-docs) docs=disabled @@ -972,7 +968,6 @@ cat << EOF bsd-user all BSD usermode emulation targets pie Position Independent Executables debug-tcg TCG debugging (default is disabled) - debug-info debugging information NOTE: The object files are built at the place where configure is launched EOF diff --git a/scripts/meson-buildoptions.py b/scripts/meson-buildoptions.py index 4c7f13fdfc40..8d2e526132ae 100755 --- a/scripts/meson-buildoptions.py +++ b/scripts/meson-buildoptions.py @@ -36,6 +36,7 @@ "b_coverage": "gcov", "b_lto": "lto", "coroutine_backend": "with-coroutine", + "debug": "debug-info", "malloc": "enable-malloc", "pkgversion": "with-pkgversion", "qemu_firmwarepath": "firmwarepath", @@ -47,6 +48,7 @@ "b_coverage", "b_lto", "datadir", + "debug", "includedir", "libdir", "libexecdir", diff --git a/scripts/meson-buildoptions.sh b/scripts/meson-buildoptions.sh index 28b6e8385bc1..d6223c5acdd1 100644 --- a/scripts/meson-buildoptions.sh +++ b/scripts/meson-buildoptions.sh @@ -11,6 +11,7 @@ meson_options_help() { printf "%s\n" ' affects only QEMU, not tools like qemu-img)' printf "%s\n" ' --datadir=VALUE Data file directory [share]' printf "%s\n" ' --disable-coroutine-pool coroutine freelist (better performance)' + printf "%s\n" ' --disable-debug-info Enable debug symbols and other information' printf "%s\n" ' --disable-hexagon-idef-parser' printf "%s\n" ' use idef-parser to automatically generate TCG' printf "%s\n" ' code for the Hexagon frontend' @@ -263,6 +264,8 @@ _meson_option_parse() { --datadir=*) quote_sh "-Ddatadir=$2" ;; --enable-dbus-display) printf "%s" -Ddbus_display=enabled ;; --disable-dbus-display) printf "%s" -Ddbus_display=disabled ;; + --enable-debug-info) printf "%s" -Ddebug=true ;; + --disable-debug-info) printf "%s" -Ddebug=false ;; --enable-debug-graph-lock) printf "%s" -Ddebug_graph_lock=true ;; --disable-debug-graph-lock) printf "%s" -Ddebug_graph_lock=false ;; --enable-debug-mutex) printf "%s" -Ddebug_mutex=true ;; From patchwork Wed May 17 17:45:15 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Paolo Bonzini X-Patchwork-Id: 1782844 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=nongnu.org (client-ip=209.51.188.17; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=) Authentication-Results: legolas.ozlabs.org; dkim=pass (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=C4r8ENdC; dkim-atps=neutral Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-ECDSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4QM0xX4P69z20dX for ; Thu, 18 May 2023 03:50:08 +1000 (AEST) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1pzLGG-0003yg-VU; Wed, 17 May 2023 13:48:29 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pzLF9-0002kJ-AO for qemu-devel@nongnu.org; Wed, 17 May 2023 13:47:22 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.129.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pzLF7-0004dk-Ey for qemu-devel@nongnu.org; Wed, 17 May 2023 13:47:18 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1684345635; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=JI3T87pJOqojWyuspcbpOcfDTh2nH5bgx4dRWFwMvSc=; b=C4r8ENdCYkc9qseI+5SsAWPebsck+aNMt3zggvrGiIjjGGCZkNpreq/tbd6yIsp4nvy3GJ cW8p0b0foUsx7qutrHihd17aJMAKHfQ768JXdPuuWUpZF6Jw63ZWgTdRDsVNHaBm1Ij0WO //gCQSWJS1pSoDUeCWXGjzzbwB5CciQ= Received: from mail-wr1-f72.google.com (mail-wr1-f72.google.com [209.85.221.72]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-113-ALBaLvZPNyyRmSgLFZJagw-1; Wed, 17 May 2023 13:47:06 -0400 X-MC-Unique: ALBaLvZPNyyRmSgLFZJagw-1 Received: by mail-wr1-f72.google.com with SMTP id ffacd0b85a97d-3093b0cf714so448892f8f.2 for ; Wed, 17 May 2023 10:47:06 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1684345625; x=1686937625; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=JI3T87pJOqojWyuspcbpOcfDTh2nH5bgx4dRWFwMvSc=; b=iU03m9P86gIGppOLVAUhLB4QLUIlAwmiKsJU0cCWdNX8nN7+SmhGPgpw+Pg6fZwMMs 2WocI71q4O1v4FQZjYY3oSA3UZMxBNmKSDb9oYyjPSbg6J/DYz1ysyO4TJ9vW0z7r8xR oEgGgfvzWQcHdMelYb+QtvEmMjz2IkU+Y4MaXDEkevUdZEWusT6sPq0R5Uk3tkrZ1/YA 2HZSm5TmaXY9XzhAg6vethOy/3FumOENH8BlBTJS7Bol8LiYM49YtnKKouxbgLSRMmMX NEtvJ1394VYuvQO7sC4f2KiNuWWwfhby06KASx+i1zBKpz7Bqsco5ECd9rdVr1uMUjn7 RbQA== X-Gm-Message-State: AC+VfDzrUOfhVwkluHtmt9vnhhahcwdFjiT59WDvtlm6xjXI4Ckq5g/7 4yTuQHJ2drLEbGHg3mIhhJ0I507nVIxTearmGyFKjxiPkkU08hoGtGfrdmMORc+x/qB5KAxRT6J jcHjA1s9jRSKmk0PTmW6oGIEyFOQwXbUtpiTxmaEuQ5cg7Tw3qW2YsOemleNXH5CeiodEc0q6lj Y= X-Received: by 2002:adf:fa03:0:b0:307:a6bc:3baf with SMTP id m3-20020adffa03000000b00307a6bc3bafmr1358519wrr.56.1684345625110; Wed, 17 May 2023 10:47:05 -0700 (PDT) X-Google-Smtp-Source: ACHHUZ7BhyD7ejqKYQlz3ZZ/grl+hZasgEmg76S/x5wUhhbqPYilsHIhV1c+Sorg8idHfvpEmXu4fg== X-Received: by 2002:adf:fa03:0:b0:307:a6bc:3baf with SMTP id m3-20020adffa03000000b00307a6bc3bafmr1358510wrr.56.1684345624776; Wed, 17 May 2023 10:47:04 -0700 (PDT) Received: from [192.168.10.118] ([2001:b07:6468:f312:9af8:e5f5:7516:fa89]) by smtp.gmail.com with ESMTPSA id a8-20020a056000100800b00307d20546e6sm3714622wrx.27.2023.05.17.10.47.04 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 17 May 2023 10:47:04 -0700 (PDT) From: Paolo Bonzini To: qemu-devel@nongnu.org Subject: [PULL 63/68] configure: remove compiler sanity check Date: Wed, 17 May 2023 19:45:15 +0200 Message-Id: <20230517174520.887405-64-pbonzini@redhat.com> X-Mailer: git-send-email 2.40.1 In-Reply-To: <20230517174520.887405-1-pbonzini@redhat.com> References: <20230517174520.887405-1-pbonzini@redhat.com> MIME-Version: 1.0 Received-SPF: pass client-ip=170.10.129.124; envelope-from=pbonzini@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org The comment is not correct anymore, in that the usability test for the compiler and linker are done after probing $cpu, and Meson will redo them anyway. Signed-off-by: Paolo Bonzini --- configure | 14 -------------- 1 file changed, 14 deletions(-) diff --git a/configure b/configure index b2172b7b938e..4a986e6fa89c 100755 --- a/configure +++ b/configure @@ -1090,20 +1090,6 @@ if test -z "$ninja"; then fi fi -# Check that the C compiler works. Doing this here before testing -# the host CPU ensures that we had a valid CC to autodetect the -# $cpu var (and we should bail right here if that's not the case). -# It also allows the help message to be printed without a CC. -write_c_skeleton; -if compile_object ; then - : C compiler works ok -else - error_exit "\"$cc\" either does not exist or does not work" -fi -if ! compile_prog ; then - error_exit "\"$cc\" cannot build an executable (is your linker broken?)" -fi - # Consult white-list to determine whether to enable werror # by default. Only enable by default for git builds if test -z "$werror" ; then From patchwork Wed May 17 17:45:16 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Paolo Bonzini X-Patchwork-Id: 1782883 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=nongnu.org (client-ip=209.51.188.17; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=) Authentication-Results: legolas.ozlabs.org; dkim=pass (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=WaYCrJf5; dkim-atps=neutral Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-ECDSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4QM1Dz5znQz20dn for ; Thu, 18 May 2023 04:03:31 +1000 (AEST) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1pzLH9-0005Wn-1o; Wed, 17 May 2023 13:49:23 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pzLF1-00029M-OO for qemu-devel@nongnu.org; Wed, 17 May 2023 13:47:11 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pzLEz-0004ZY-UX for qemu-devel@nongnu.org; Wed, 17 May 2023 13:47:11 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1684345629; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=xPou24ScysZCFeYkKf5HYVYev95mGveP+l81GewaT7I=; b=WaYCrJf55ACVqsAdXN4OWapjUyfBGxDrRuMeVduFRAfzkZp2loRlQ2YjkJv5ttQQyLYNQr fPk+gEtEuR8M/5rYo0azDT49/MuaRNhjnC/ZcoRagOx+Br6XQ01fj4WE+umFEfieH3wpDE BGvzRsAlucYDK+jIZkjR1AHPjB8pqC4= Received: from mail-wr1-f71.google.com (mail-wr1-f71.google.com [209.85.221.71]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-616-_g_03CvhOAW8Y5Csm6wBrQ-1; Wed, 17 May 2023 13:47:08 -0400 X-MC-Unique: _g_03CvhOAW8Y5Csm6wBrQ-1 Received: by mail-wr1-f71.google.com with SMTP id ffacd0b85a97d-3062dedf7d9so753410f8f.2 for ; Wed, 17 May 2023 10:47:07 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1684345626; x=1686937626; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=xPou24ScysZCFeYkKf5HYVYev95mGveP+l81GewaT7I=; b=hxcVBCKr4vNg69onB9WEqXq3Vn/DYckEeb+KWEdGGoqPuebgqJSpDES4x8od8Mr3gK h9XqTwUzM4ByPQlc02wEpowiW8WucVU8fdAv8u76qKAPzSYLJX1wGMzfIMhhdrMhcc6J NQq+8QJdGyScf6bYEXrmV9b+CstSZDxPH0r7GVGGsJ8Cv9C+VYVuMi4/iO4KWnBrCovz L6vPM+/ju7dBdC5n4mAdcM2awQUbruJHWOh1QdkUrh/s58BxQwvh93ToPI7qf7WxVdtt OVyEK7jtnnxWxcsWOc+vbi8ES9G+kGwO5ZCVjuEbvCxVVGfPFoLw45luCNIlrT67NFcC ajJw== X-Gm-Message-State: AC+VfDz7iGeXhpU5AobzO7XgPGreqXczFQ4WYK8sVMyyZ5ofpz4KCrqt 1PiI/AbvTqxPIdWGqVKd7CYzI3WXWIAbWALlFG8pfIYA2f/zTbUpAc8Nk1VZ6X3BxvmbS0t3xic 3yX8ARkh4D7MoD266uPu7Zy/bwCJxSJWxfk/9uU1vpD2wZ+B85f/3amgRefjpaM5UYhb5g+J1Er s= X-Received: by 2002:a5d:694a:0:b0:309:44ed:ccff with SMTP id r10-20020a5d694a000000b0030944edccffmr944218wrw.1.1684345626502; Wed, 17 May 2023 10:47:06 -0700 (PDT) X-Google-Smtp-Source: ACHHUZ6dSrk1obeUsN4oXWhzS/eET2HCf8sVX4eiabZWU1wzr6KkrDz0+XqN8VgX307ySNpQ99gKtQ== X-Received: by 2002:a5d:694a:0:b0:309:44ed:ccff with SMTP id r10-20020a5d694a000000b0030944edccffmr944209wrw.1.1684345626150; Wed, 17 May 2023 10:47:06 -0700 (PDT) Received: from [192.168.10.118] ([2001:b07:6468:f312:9af8:e5f5:7516:fa89]) by smtp.gmail.com with ESMTPSA id m7-20020a5d6a07000000b003062c0ef959sm3325432wru.69.2023.05.17.10.47.05 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 17 May 2023 10:47:05 -0700 (PDT) From: Paolo Bonzini To: qemu-devel@nongnu.org Cc: =?utf-8?q?Marc-Andr=C3=A9_Lureau?= Subject: [PULL 64/68] configure: do not rerun the tests with -Werror Date: Wed, 17 May 2023 19:45:16 +0200 Message-Id: <20230517174520.887405-65-pbonzini@redhat.com> X-Mailer: git-send-email 2.40.1 In-Reply-To: <20230517174520.887405-1-pbonzini@redhat.com> References: <20230517174520.887405-1-pbonzini@redhat.com> MIME-Version: 1.0 Received-SPF: pass client-ip=170.10.133.124; envelope-from=pbonzini@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Tests run in configure are pretty trivial at this point, so do not bother with the extra complication of running tests both with and without -Werror. Reviewed-by: Marc-AndrĂ© Lureau Signed-off-by: Paolo Bonzini --- configure | 35 +---------------------------------- 1 file changed, 1 insertion(+), 34 deletions(-) diff --git a/configure b/configure index 4a986e6fa89c..d0febe7cdfea 100755 --- a/configure +++ b/configure @@ -124,41 +124,8 @@ lines: ${BASH_LINENO[*]}" $compiler "$@" >> config.log 2>&1 || return $? } -do_compiler_werror() { - # Run the compiler, capturing its output to the log. First argument - # is compiler binary to execute. - compiler="$1" - shift - if test -n "$BASH_VERSION"; then eval ' - echo >>config.log " -funcs: ${FUNCNAME[*]} -lines: ${BASH_LINENO[*]}" - '; fi - echo $compiler "$@" >> config.log - $compiler "$@" >> config.log 2>&1 || return $? - # Test passed. If this is an --enable-werror build, rerun - # the test with -Werror and bail out if it fails. This - # makes warning-generating-errors in configure test code - # obvious to developers. - if test "$werror" != "yes"; then - return 0 - fi - # Don't bother rerunning the compile if we were already using -Werror - case "$*" in - *-Werror*) - return 0 - ;; - esac - echo $compiler -Werror "$@" >> config.log - $compiler -Werror "$@" >> config.log 2>&1 && return $? - error_exit "configure test passed without -Werror but failed with -Werror." \ - "This is probably a bug in the configure script. The failing command" \ - "will be at the bottom of config.log." \ - "You can run configure with --disable-werror to bypass this check." -} - do_cc() { - do_compiler_werror "$cc" $CPU_CFLAGS "$@" + do_compiler "$cc" $CPU_CFLAGS "$@" } compile_object() { From patchwork Wed May 17 17:45:17 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Paolo Bonzini X-Patchwork-Id: 1782857 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=nongnu.org (client-ip=209.51.188.17; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=) Authentication-Results: legolas.ozlabs.org; dkim=pass (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=X9/FzK+O; dkim-atps=neutral Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-ECDSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4QM12r5S3Lz20KF for ; Thu, 18 May 2023 03:54:44 +1000 (AEST) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1pzLGG-0003vC-8t; Wed, 17 May 2023 13:48:28 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pzLF3-0002FB-7c for qemu-devel@nongnu.org; Wed, 17 May 2023 13:47:13 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pzLF1-0004am-Iz for qemu-devel@nongnu.org; Wed, 17 May 2023 13:47:12 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1684345631; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=7L8ko30fbEd8tn2dVN6/DcnAocK307mvGHN2bv7IKMc=; b=X9/FzK+OaZ+vTeOKouEx9CiWeZLbx5mW02K5QF+5vJCMjB/Szp9F2sp0VYm4/lTvZl0vZI Xckx3QxyXuvDmpAVE5LPPnK4Z2s8s5Qn+nMvM9vKidpdSxhYok6YX8/prU9sXaKnKPJPWv cCdIWBmWBLXxCP/f46PLEjyXun5Lcds= Received: from mail-wm1-f72.google.com (mail-wm1-f72.google.com [209.85.128.72]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-640-mYzqcgXBOxSAlsEcgSKaeQ-1; Wed, 17 May 2023 13:47:09 -0400 X-MC-Unique: mYzqcgXBOxSAlsEcgSKaeQ-1 Received: by mail-wm1-f72.google.com with SMTP id 5b1f17b1804b1-3f419a053c5so1709615e9.1 for ; Wed, 17 May 2023 10:47:09 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1684345628; x=1686937628; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=7L8ko30fbEd8tn2dVN6/DcnAocK307mvGHN2bv7IKMc=; b=VBsyJkrkzbvSHoucdusbeui7Dnf+FXzKNOmckYDU8sK2ePmt4Rli+wh9OLtYvno/rP c0MijjMTUybSLUUniWAMH3gOQIzrxbpqfXw14sVnGKX2Me1eq+AodJT76176bUZhUg1j vDs9EH1FcZhAgx4xOLCnfV6coWe/4XPbogQt6TFSCfF6U6Dyz6BfVVilr1MtEEWv9AZe 8RhB1G/ZAifMwNUpKyxS9H/JCKLmktv5FrlophX5TPmqG4Qm6Q+ipsF0OM9zZVFSPX+T BWX4/r2sRyqR571cQx6kfHpSbq7y6LSLAAqV1wC+H/TpbPyQ6vpHWJ3jq8WMdXVXzoYl G8eA== X-Gm-Message-State: AC+VfDz4uSRAQf3xaWOgi4oK6+y4FMmig4d+NRyq/KbQV0r4LY0FfABj f1AanjPbGJM1ZquRV5VoR3QKRE8EfZUlrw7AXenVRs6cvsyf71NJKmyJeJE/1Mjmj2+leBG09eh zfckXYw9Uku5Oc+/sPi0ZOE6tahmPYBqAvk74ZJaHjZBF1ECdQ47T6g9kYTT8WC173jKkNP8Y+h 8= X-Received: by 2002:a7b:c3ce:0:b0:3f4:2174:b28c with SMTP id t14-20020a7bc3ce000000b003f42174b28cmr2250983wmj.14.1684345627965; Wed, 17 May 2023 10:47:07 -0700 (PDT) X-Google-Smtp-Source: ACHHUZ7B2zPC5ZLx4VG0qjVIDkKcMn9GjXNmG3lsp9UYwxJ+WVxHGrPSjLrnuXpuCmadTH3Z17PYNA== X-Received: by 2002:a7b:c3ce:0:b0:3f4:2174:b28c with SMTP id t14-20020a7bc3ce000000b003f42174b28cmr2250972wmj.14.1684345627628; Wed, 17 May 2023 10:47:07 -0700 (PDT) Received: from [192.168.10.118] ([2001:b07:6468:f312:9af8:e5f5:7516:fa89]) by smtp.gmail.com with ESMTPSA id c22-20020a7bc856000000b003f42cc3262asm2863586wml.34.2023.05.17.10.47.06 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 17 May 2023 10:47:06 -0700 (PDT) From: Paolo Bonzini To: qemu-devel@nongnu.org Subject: [PULL 65/68] configure: remove unnecessary mkdir Date: Wed, 17 May 2023 19:45:17 +0200 Message-Id: <20230517174520.887405-66-pbonzini@redhat.com> X-Mailer: git-send-email 2.40.1 In-Reply-To: <20230517174520.887405-1-pbonzini@redhat.com> References: <20230517174520.887405-1-pbonzini@redhat.com> MIME-Version: 1.0 Received-SPF: pass client-ip=170.10.133.124; envelope-from=pbonzini@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org It is taken care of by the symlink shell function. Signed-off-by: Paolo Bonzini --- configure | 1 - 1 file changed, 1 deletion(-) diff --git a/configure b/configure index d0febe7cdfea..2d6ff2d6cef6 100755 --- a/configure +++ b/configure @@ -1673,7 +1673,6 @@ LINKS="$LINKS python" LINKS="$LINKS contrib/plugins/Makefile " for f in $LINKS ; do if [ -e "$source_path/$f" ]; then - mkdir -p "$(dirname ./"$f")" symlink "$source_path/$f" "$f" fi done From patchwork Wed May 17 17:45:18 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Paolo Bonzini X-Patchwork-Id: 1782874 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=nongnu.org (client-ip=209.51.188.17; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=) Authentication-Results: legolas.ozlabs.org; dkim=pass (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=E28uFyfV; dkim-atps=neutral Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-ECDSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4QM18459mXz20dX for ; Thu, 18 May 2023 03:59:16 +1000 (AEST) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1pzLH7-0005Tn-To; Wed, 17 May 2023 13:49:21 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pzLF4-0002LE-FC for qemu-devel@nongnu.org; Wed, 17 May 2023 13:47:14 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pzLF2-0004bj-NU for qemu-devel@nongnu.org; Wed, 17 May 2023 13:47:14 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1684345632; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=QBLkLIPImIhiMXXreUR0S6NO4vQOLpLc4SPHm9Q7bTE=; b=E28uFyfVN4b6MnBqTXPDvhao5H/18qnRwJhpVXIa8mvIrXaP61UCZpio+z+Rc8g+CrMklg uWH8VqHJ7EjxCKl+Hdm8EFyNMm47pw/8ZQ5+P1ow/8URptG7FoUQiS++6amcsb0eRDRIEY kGQlAhiVbo2ciCOshIRUZT535rC5PFs= Received: from mail-wm1-f72.google.com (mail-wm1-f72.google.com [209.85.128.72]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-282-ho86Qy0iOgaKgM-f19Y7mA-1; Wed, 17 May 2023 13:47:10 -0400 X-MC-Unique: ho86Qy0iOgaKgM-f19Y7mA-1 Received: by mail-wm1-f72.google.com with SMTP id 5b1f17b1804b1-3f453ff4cdfso4731635e9.2 for ; Wed, 17 May 2023 10:47:10 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1684345629; x=1686937629; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=QBLkLIPImIhiMXXreUR0S6NO4vQOLpLc4SPHm9Q7bTE=; b=dz1AC8V7XxadZ8roNP+uzf5UxA8Pe3aWkZJ9LOfD2d0DIH2fWQ2v6X+CAhd5eR6qpS 1PORf+Ke4srZ7rv88GfL/mfkxQX0Mt490xWrxFGwEV4LU3EA5ukOAQBK4xYJS1wYCoyw 9K+gDnwL1+rRWorAWROBBwTffiu3+gceEPbB2fzde9fUMLKTaVNi0imsiSUvj9zJkUG+ i64AuH1rRkc+U2FDA0Ycpa87KZci6GslY7I7eZ3BWHsGZKdrZE8F85SHr3Tsfn/sU0qi 5WEUTxCfC5OLHirKqMK10Z4xChd910GEGoSKQ/OknUmJBPSs+2yWO7pWGCZDd857NP5h PtRw== X-Gm-Message-State: AC+VfDzpggqhj1uR9klIWoYIemO2mW4GCjhArkOj4JJkONhEeVS2jk5E OFzcpp5nfZVW/pLy1dXSMfXW2Bsf2G57/Ta3Jyyi6K7EVWSbJxb7xYgB/mnmV+LQPg8Pz/Xytby TFg1f8vzUcJ89NziyfpUp/ytX3SXs8cxzFYhFpcMi4Fw+CozIM57HncHNeIngcc2WXmSZHCE94/ s= X-Received: by 2002:a7b:c014:0:b0:3f5:f52:beb1 with SMTP id c20-20020a7bc014000000b003f50f52beb1mr6495845wmb.12.1684345629295; Wed, 17 May 2023 10:47:09 -0700 (PDT) X-Google-Smtp-Source: ACHHUZ7TFsNgingm7QcvhO+D204bxDN0/OPb2UjK3hfUERNf5YiwAA5nNAO8z7874ZLMkPGhByxDcw== X-Received: by 2002:a7b:c014:0:b0:3f5:f52:beb1 with SMTP id c20-20020a7bc014000000b003f50f52beb1mr6495836wmb.12.1684345628943; Wed, 17 May 2023 10:47:08 -0700 (PDT) Received: from [192.168.10.118] ([2001:b07:6468:f312:9af8:e5f5:7516:fa89]) by smtp.gmail.com with ESMTPSA id i9-20020adffc09000000b0030497b3224bsm3393796wrr.64.2023.05.17.10.47.08 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 17 May 2023 10:47:08 -0700 (PDT) From: Paolo Bonzini To: qemu-devel@nongnu.org Subject: [PULL 66/68] configure: reorder option parsing code Date: Wed, 17 May 2023 19:45:18 +0200 Message-Id: <20230517174520.887405-67-pbonzini@redhat.com> X-Mailer: git-send-email 2.40.1 In-Reply-To: <20230517174520.887405-1-pbonzini@redhat.com> References: <20230517174520.887405-1-pbonzini@redhat.com> MIME-Version: 1.0 Received-SPF: pass client-ip=170.10.133.124; envelope-from=pbonzini@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Move some variable assignments around for clarity and to remove one of three loops on the command line arguments. Signed-off-by: Paolo Bonzini --- configure | 121 ++++++++++++++++++++++++++---------------------------- 1 file changed, 58 insertions(+), 63 deletions(-) diff --git a/configure b/configure index 2d6ff2d6cef6..9dc17aaa7c86 100755 --- a/configure +++ b/configure @@ -177,29 +177,18 @@ then error_exit "main directory cannot contain spaces nor colons" fi +# parse CC options first; some compiler tests are used to establish +# some defaults, based on the host environment + # default parameters cpu="" -static="no" cross_compile="no" cross_prefix="" host_cc="cc" -use_containers="yes" -gdb_bin=$(command -v "gdb-multiarch" || command -v "gdb") -gdb_arches="" - -if test -e "$source_path/.git" -then - git_submodules_action="update" -else - git_submodules_action="ignore" -fi - -git_submodules="ui/keycodemapdb" -git="git" - -# Don't accept a target_list environment variable. -unset target_list -unset target_list_exclude +EXTRA_CFLAGS="" +EXTRA_CXXFLAGS="" +EXTRA_OBJCFLAGS="" +EXTRA_LDFLAGS="" # Default value for a variable defining feature "foo". # * foo="no" feature will only be used if --enable-foo arg is given @@ -212,49 +201,8 @@ unset target_list_exclude # Always add --enable-foo and --disable-foo command line args. # Distributions want to ensure that several features are compiled in, and it # is impossible without a --enable-foo that exits if a feature is not found. - default_feature="" -# parse CC options second -for opt do - optarg=$(expr "x$opt" : 'x[^=]*=\(.*\)') - case "$opt" in - --without-default-features) - default_feature="no" - ;; - esac -done -EXTRA_CFLAGS="" -EXTRA_CXXFLAGS="" -EXTRA_OBJCFLAGS="" -EXTRA_LDFLAGS="" - -debug_tcg="no" -docs="auto" -EXESUF="" -prefix="/usr/local" -qemu_suffix="qemu" -softmmu="yes" -linux_user="" -bsd_user="" -pie="" -plugins="$default_feature" -ninja="" -bindir="bin" -skip_meson=no -vfio_user_server="disabled" - -# The following Meson options are handled manually (still they -# are included in the automatically generated help message) - -# 1. Track which submodules are needed -fdt="auto" - -# 2. Automatically enable/disable other options -tcg="auto" -cfi="false" - -# parse CC options second for opt do optarg=$(expr "x$opt" : 'x[^=]*=\(.*\)') case "$opt" in @@ -293,9 +241,60 @@ for opt do --cross-prefix-*) cc_arch=${opt#--cross-prefix-}; cc_arch=${cc_arch%%=*} eval "cross_prefix_${cc_arch}=\$optarg" ;; + --without-default-features) default_feature="no" + ;; esac done + +if test -e "$source_path/.git" +then + git_submodules_action="update" +else + git_submodules_action="ignore" +fi + +git_submodules="ui/keycodemapdb" +git="git" +debug_tcg="no" +docs="auto" +EXESUF="" +prefix="/usr/local" +qemu_suffix="qemu" +softmmu="yes" +linux_user="" +bsd_user="" +plugins="$default_feature" +ninja="" +python= +pypi="enabled" +bindir="bin" +skip_meson=no +vfio_user_server="disabled" +use_containers="yes" +gdb_bin=$(command -v "gdb-multiarch" || command -v "gdb") +gdb_arches="" +werror="" + +# Don't accept a target_list environment variable. +unset target_list +unset target_list_exclude + +# The following Meson options are handled manually (still they +# are included in the automatically generated help message) + +# 1. Track which submodules are needed +fdt="auto" + +# 2. Automatically enable/disable other options +tcg="auto" +cfi="false" + +# 3. Need to check for -static-pie before Meson runs. Also, +# Meson has PIE as a boolean rather than enabled/disabled/auto. +pie="" +static="no" + # Preferred compiler: # ${CC} (if set) # ${cross_prefix}gcc (if cross-prefix specified) @@ -536,8 +535,6 @@ check_py_version() { "$1" -c 'import sys; sys.exit(sys.version_info < (3,7))' } -python= -pypi="enabled" first_python= if test -z "${PYTHON}"; then # A bare 'python' is traditionally python 2.x, but some distros @@ -584,8 +581,6 @@ if test "$mingw32" = "yes" ; then qemu_suffix="" fi -werror="" - meson_option_build_array() { printf '[' (if test "$targetos" = windows; then From patchwork Wed May 17 17:45:19 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Paolo Bonzini X-Patchwork-Id: 1782886 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=nongnu.org (client-ip=209.51.188.17; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=) Authentication-Results: legolas.ozlabs.org; dkim=pass (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=WARcQkVL; dkim-atps=neutral Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-ECDSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4QM1FJ57Mkz20dq for ; Thu, 18 May 2023 04:03:48 +1000 (AEST) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1pzLH5-0005Kt-6b; Wed, 17 May 2023 13:49:19 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pzLFD-0002si-EM for qemu-devel@nongnu.org; Wed, 17 May 2023 13:47:25 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pzLFC-0004f2-2Z for qemu-devel@nongnu.org; Wed, 17 May 2023 13:47:23 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1684345641; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=WjLgdJJv5pfwWhI+KcQ/W+YlH4SCMKiudSw5H03DTe0=; b=WARcQkVLhqSg9BXdAWgTkG7TJsJ32Wozy+1cuKhM6ZpU7pQ70zi79Mkr/iqJAsGPFJGGj+ pBwQLvdvNxi8co59q8HDeriMlWkljUVaR7ksnJ1WatvQNFq6HYFrT3RVo+b7JrXGLW7Ik6 jYI/3S70ZU9jnS83tEMLi8zxdKvd5kE= Received: from mail-wm1-f70.google.com (mail-wm1-f70.google.com [209.85.128.70]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-380-miTNelp9P2K7YzlAopVjBw-1; Wed, 17 May 2023 13:47:12 -0400 X-MC-Unique: miTNelp9P2K7YzlAopVjBw-1 Received: by mail-wm1-f70.google.com with SMTP id 5b1f17b1804b1-3f42b36733aso4441825e9.3 for ; Wed, 17 May 2023 10:47:11 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1684345630; x=1686937630; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=WjLgdJJv5pfwWhI+KcQ/W+YlH4SCMKiudSw5H03DTe0=; b=laytU6aUBqV98Gf2ANxkrDyDS8nTI3mK1vQ1QnOuwXB+ZZ3HAw9/KKKtYAdYoHUNcf ALKozcHDJtHadAA4XiN4zxPgYd2VeTqkGxWnPank4mF6Vq3GjAcLcBDdDaOUSVgnD2/9 M/xXLie8J7etdtOGaAtCdksSNjTBJA/Kv0IZP2Hot0kQR14t7ZgRKgzBF7efBDptchxx lvnL7pXcWKv04h/yKw9KHoS1CMlr5hNt/LLIDZicCcfKEQAqq+Jv1uAEJiLdoj1VqL8o BBH46n6U4n3ThWZgF7uT8NOe9FKU7rpmfTqaHhUsli/vRBoMsgSKAxI9Iz4yV1WtdRz+ MsZA== X-Gm-Message-State: AC+VfDxa0M6UK/otU1gJYqBb2sHhR8NcqzFr/RewyVSgU+/u6BOhv46F bQUhS7aC0kbwuCy9T54MIMOMXxbsQ63umZOeHPcKgjeFoeQDOBYlCzacpfL8WtRTuZZ97rQkbXl ZbbKgeQikjI0vBp1kW6wwVCz+uSl7krBxWVs0RWwUmtO/v+pnOlDpnOe7S7SNarohVBGwg+RZ7O 4= X-Received: by 2002:a1c:4b0c:0:b0:3f4:20bd:ba4b with SMTP id y12-20020a1c4b0c000000b003f420bdba4bmr26054080wma.21.1684345630580; Wed, 17 May 2023 10:47:10 -0700 (PDT) X-Google-Smtp-Source: ACHHUZ73loZc611ydlMbh0vrMKVcshfTvaJRulfmeGLwyvCS/R8nd1Gj6k73xY4x/uHQWksTHRtqZw== X-Received: by 2002:a1c:4b0c:0:b0:3f4:20bd:ba4b with SMTP id y12-20020a1c4b0c000000b003f420bdba4bmr26054071wma.21.1684345630218; Wed, 17 May 2023 10:47:10 -0700 (PDT) Received: from [192.168.10.118] ([2001:b07:6468:f312:9af8:e5f5:7516:fa89]) by smtp.gmail.com with ESMTPSA id v5-20020a1cf705000000b003f4f1b884b3sm2842919wmh.20.2023.05.17.10.47.09 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 17 May 2023 10:47:09 -0700 (PDT) From: Paolo Bonzini To: qemu-devel@nongnu.org Subject: [PULL 67/68] configure: remove unnecessary check Date: Wed, 17 May 2023 19:45:19 +0200 Message-Id: <20230517174520.887405-68-pbonzini@redhat.com> X-Mailer: git-send-email 2.40.1 In-Reply-To: <20230517174520.887405-1-pbonzini@redhat.com> References: <20230517174520.887405-1-pbonzini@redhat.com> MIME-Version: 1.0 Received-SPF: pass client-ip=170.10.133.124; envelope-from=pbonzini@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org All calls to probe_target_compiler are conditioned on some "have_target" invocation, or inside a loop on target_list. Therefore there is no issue with building unnecessary firmware images and tests. Signed-off-by: Paolo Bonzini --- configure | 7 ------- 1 file changed, 7 deletions(-) diff --git a/configure b/configure index 9dc17aaa7c86..7135251fdcd8 100755 --- a/configure +++ b/configure @@ -1324,13 +1324,6 @@ probe_target_compiler() { container_cross_ranlib= container_cross_strip= - # We shall skip configuring the target compiler if the user didn't - # bother enabling an appropriate guest. This avoids building - # extraneous firmware images and tests. - if test "${target_list#*$1}" = "$1"; then - return 1 - fi - target_arch=${1%%-*} case $target_arch in aarch64) container_hosts="x86_64 aarch64" ;; From patchwork Wed May 17 17:45:20 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Paolo Bonzini X-Patchwork-Id: 1782871 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=nongnu.org (client-ip=209.51.188.17; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=) Authentication-Results: legolas.ozlabs.org; dkim=pass (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=KzRRfYqj; dkim-atps=neutral Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-ECDSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4QM16Y53BZz20dX for ; Thu, 18 May 2023 03:57:57 +1000 (AEST) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1pzLGp-0004lH-SD; Wed, 17 May 2023 13:49:04 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pzLFA-0002mv-FF for qemu-devel@nongnu.org; Wed, 17 May 2023 13:47:22 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pzLF7-0004dm-EN for qemu-devel@nongnu.org; Wed, 17 May 2023 13:47:19 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1684345636; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=vpHfQMAHim+xg75TnPiP56zC+UTGEuoT9DPQNdibHlI=; b=KzRRfYqjbEerpJHd9O/v8XRp+GmPNFcFl+GmUIQacqVpUcsbVgd4OZiVSy01AM3LG5HvMp 2hdunkb4Z3tvueejuF5uXd/HFI59dy1eHPAvTrf/GJN2Dzm4VnGIsLBYsp8ogtj2cXcmCT jLcxdGedJA440janeScrI6oWVQXfqls= Received: from mail-wr1-f72.google.com (mail-wr1-f72.google.com [209.85.221.72]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-194-0k-1oJAjNJ-1NngqY2LkPg-1; Wed, 17 May 2023 13:47:14 -0400 X-MC-Unique: 0k-1oJAjNJ-1NngqY2LkPg-1 Received: by mail-wr1-f72.google.com with SMTP id ffacd0b85a97d-2f2981b8364so698571f8f.1 for ; Wed, 17 May 2023 10:47:13 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1684345632; x=1686937632; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=vpHfQMAHim+xg75TnPiP56zC+UTGEuoT9DPQNdibHlI=; b=F4+mdF6MbD9jgncpm8PGszHt76Q9z1kF+Hnzel9VZM+j6vYSXlQ956KkRmi6QZCSgA TiEmmd+DVbwNJa/CwPNrmyt8fcwgtxk+5fkMtRDoqytp3fAQyjhqBxaIZN0lZPd0vSYz PdNDyEbD4cK5AhAVViqhIR3Nqn6Qp3WFmIyW1Ve64I9zWHgdOHXxKCWzAwxGswO3w/YK DZAIc6APCzWoAw4QQe/pR2SYOJJVtinMXNW96J4nbpjATPSCP66mFgN72NnjMWrzQvaW 6ElVz7vN7t5ZYAvvmLlzF0UXyh0G6g20Sz4AHVthuJK3vIvUvvaCIHNEk2ESUP1Rw71b 6miQ== X-Gm-Message-State: AC+VfDwkKaikljVl2UaGphYJz2HpYdg1lQKFn7epyekMUU+uaP3ZNigL 1qbO5e3atQfzwO+9MDuRJaN9C+6XUzdDLoaXcUisb2TpaaFLYO+KIJjhmdkgP1SoaQti3a5vYaG 71aPQ3kXmm7YFXFuIZXAjLkQ9zS1UOzh8IpoFtlz1J7bGo29va6ECXrhQD2cHyq31QXtDZn57aB Q= X-Received: by 2002:adf:ef08:0:b0:306:2e04:5925 with SMTP id e8-20020adfef08000000b003062e045925mr1364142wro.17.1684345632325; Wed, 17 May 2023 10:47:12 -0700 (PDT) X-Google-Smtp-Source: ACHHUZ5Xqh+5FHntHnrrcP0rby7kI/Q/NNp7uM3zSonloT2clrmLEyfbFe4uDcBmXJbfNzm05u6kbw== X-Received: by 2002:adf:ef08:0:b0:306:2e04:5925 with SMTP id e8-20020adfef08000000b003062e045925mr1364114wro.17.1684345631534; Wed, 17 May 2023 10:47:11 -0700 (PDT) Received: from [192.168.10.118] ([2001:b07:6468:f312:9af8:e5f5:7516:fa89]) by smtp.gmail.com with ESMTPSA id u14-20020adfdd4e000000b002f22c44e974sm3393883wrm.102.2023.05.17.10.47.10 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 17 May 2023 10:47:10 -0700 (PDT) From: Paolo Bonzini To: qemu-devel@nongnu.org Subject: [PULL 68/68] docs/devel: update build system docs Date: Wed, 17 May 2023 19:45:20 +0200 Message-Id: <20230517174520.887405-69-pbonzini@redhat.com> X-Mailer: git-send-email 2.40.1 In-Reply-To: <20230517174520.887405-1-pbonzini@redhat.com> References: <20230517174520.887405-1-pbonzini@redhat.com> MIME-Version: 1.0 Received-SPF: pass client-ip=170.10.133.124; envelope-from=pbonzini@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org configure is only doing compiler and host setup now, so adjust the relevant documentation. It is also possible to build emulators with ninja directly if one is so inclined, so mention that as well. The Python virtual environment set up is a new major task of configure as well. Mention it in the list of produced files, while leaving it for a future patch to document how it works and how ``mkvenv ensure`` is used. Signed-off-by: Paolo Bonzini --- docs/devel/build-system.rst | 287 ++++++++++++++++++++++-------------- 1 file changed, 177 insertions(+), 110 deletions(-) diff --git a/docs/devel/build-system.rst b/docs/devel/build-system.rst index 4a733fc0a747..551c5a5ac0ae 100644 --- a/docs/devel/build-system.rst +++ b/docs/devel/build-system.rst @@ -4,30 +4,14 @@ The QEMU build system architecture This document aims to help developers understand the architecture of the QEMU build system. As with projects using GNU autotools, the QEMU build -system has two stages, first the developer runs the "configure" script +system has two stages; first the developer runs the "configure" script to determine the local build environment characteristics, then they run -"make" to build the project. There is about where the similarities with +"make" to build the project. This is about where the similarities with GNU autotools end, so try to forget what you know about them. +The two general ways to perform a build are as follows: -Stage 1: configure -================== - -The QEMU configure script is written directly in shell, and should be -compatible with any POSIX shell, hence it uses #!/bin/sh. An important -implication of this is that it is important to avoid using bash-isms on -development platforms where bash is the primary host. - -In contrast to autoconf scripts, QEMU's configure is expected to be -silent while it is checking for features. It will only display output -when an error occurs, or to show the final feature enablement summary -on completion. - -Because QEMU uses the Meson build system under the hood, only VPATH -builds are supported. There are two general ways to invoke configure & -perform a build: - - - VPATH, build artifacts outside of QEMU source tree entirely:: + - build artifacts outside of QEMU source tree entirely:: cd ../ mkdir build @@ -35,80 +19,122 @@ perform a build: ../qemu/configure make - - VPATH, build artifacts in a subdir of QEMU source tree:: + - build artifacts in a subdir of QEMU source tree:: mkdir build cd build ../configure make -The configure script automatically recognizes -command line options for which a same-named Meson option exists; -dashes in the command line are replaced with underscores. +Most of the actual build process uses Meson under the hood, therefore +build artifacts cannot be placed in the source tree itself. -Many checks on the compilation environment are still found in configure -rather than ``meson.build``, but new checks should be added directly to -``meson.build``. -Patches are also welcome to move existing checks from the configure -phase to ``meson.build``. When doing so, ensure that ``meson.build`` does -not use anymore the keys that you have removed from ``config-host.mak``. -Typically these will be replaced in ``meson.build`` by boolean variables, -``get_option('optname')`` invocations, or ``dep.found()`` expressions. -In general, the remaining checks have little or no interdependencies, -so they can be moved one by one. +Stage 1: configure +================== -Helper functions ----------------- +The configure script has five tasks: -The configure script provides a variety of helper functions to assist -developers in checking for system features: + - detect the host architecture -``do_cc $ARGS...`` - Attempt to run the system C compiler passing it $ARGS... + - list the targets for which to build emulators; the list of + targets also affects which firmware binaries and tests to build -``do_cxx $ARGS...`` - Attempt to run the system C++ compiler passing it $ARGS... + - find the compilers (native and cross) used to build executables, + firmware and tests. The results are written as either Makefile + fragments (``config-host.mak``) or a Meson machine file + (``config-meson.cross``) -``compile_object $CFLAGS`` - Attempt to compile a test program with the system C compiler using - $CFLAGS. The test program must have been previously written to a file - called $TMPC. The replacement in Meson is the compiler object ``cc``, - which has methods such as ``cc.compiles()``, - ``cc.check_header()``, ``cc.has_function()``. + - create a virtual environment in which all Python code runs during + the build, and possibly install packages into it from PyPI -``compile_prog $CFLAGS $LDFLAGS`` - Attempt to compile a test program with the system C compiler using - $CFLAGS and link it with the system linker using $LDFLAGS. The test - program must have been previously written to a file called $TMPC. - The replacement in Meson is ``cc.find_library()`` and ``cc.links()``. + - invoke Meson in the virtual environment, to perform the actual + configuration step for the emulator build + +The configure script automatically recognizes command line options for +which a same-named Meson option exists; dashes in the command line are +replaced with underscores. + +Almost all QEMU developers that need to modify the build system will +only be concerned with Meson, and therefore can skip the rest of this +section. + + +Modifying ``configure`` +----------------------- + +``configure`` is a shell script; it uses ``#!/bin/sh`` and therefore +should be compatible with any POSIX shell. It is important to avoid +using bash-isms to avoid breaking development platforms where bash is +the primary host. + +The configure script provides a variety of functions to help writing +portable shell code and providing consistent behavior across architectures +and operating systems: + +``error_exit $MESSAGE $MORE...`` + Print $MESSAGE to stderr, followed by $MORE... and then exit from the + configure script with non-zero status. ``has $COMMAND`` Determine if $COMMAND exists in the current environment, either as a shell builtin, or executable binary, returning 0 on success. The replacement in Meson is ``find_program()``. +``probe_target_compiler $TARGET`` + Detect a cross compiler and cross tools for the QEMU target $TARGET (e.g., + ``$CPU-softmmu``, ``$CPU-linux-user``, ``$CPU-bsd-user``). If a working + compiler is present, return success and set variables ``$target_cc``, + ``$target_ar``, etc. to non-empty values. + +``write_target_makefile`` + Write a Makefile fragment to stdout, exposing the result of the most + ``probe_target_compiler`` call as the usual Make variables (``CC``, + ``AR``, ``LD``, etc.). + + +Configure does not generally perform tests for compiler options beyond +basic checks to detect the host platform and ensure the compiler is +functioning. These are performed using a few more helper functions: + +``compile_object $CFLAGS`` + Attempt to compile a test program with the system C compiler using + $CFLAGS. The test program must have been previously written to a file + called $TMPC. + +``compile_prog $CFLAGS $LDFLAGS`` + Attempt to compile a test program with the system C compiler using + $CFLAGS and link it with the system linker using $LDFLAGS. The test + program must have been previously written to a file called $TMPC. + ``check_define $NAME`` - Determine if the macro $NAME is defined by the system C compiler + Determine if the macro $NAME is defined by the system C compiler. + +``do_compiler $CC $ARGS...`` + Attempt to run the C compiler $CC, passing it $ARGS... This function + does not use flags passed via options such as ``--extra-cflags``, and + therefore can be used to check for cross compilers. However, most + such checks are done at ``make`` time instead (see for example the + ``cc-option`` macro in ``pc-bios/option-rom/Makefile``). ``write_c_skeleton`` Write a minimal C program main() function to the temporary file - indicated by $TMPC + indicated by $TMPC. -``error_exit $MESSAGE $MORE...`` - Print $MESSAGE to stderr, followed by $MORE... and then exit from the - configure script with non-zero status +Python virtual environments and the QEMU build system +----------------------------------------------------- + +TBD Stage 2: Meson ============== -The Meson build system is currently used to describe the build -process for: +The Meson build system describes the build and install process for: 1) executables, which include: - - Tools - ``qemu-img``, ``qemu-nbd``, ``qga`` (guest agent), etc + - Tools - ``qemu-img``, ``qemu-nbd``, ``qemu-ga`` (guest agent), etc - System emulators - ``qemu-system-$ARCH`` @@ -118,7 +144,8 @@ process for: 2) documentation -3) ROMs, which can be either installed as binary blobs or compiled +3) ROMs, whether provided as binary blobs in the QEMU distributions + or cross compiled under the direction of the configure script 4) other data files, such as icons or desktop files @@ -273,8 +300,7 @@ system/userspace emulation target Adding checks ------------- -New checks should be added to Meson. Compiler checks can be as simple as -the following:: +Compiler checks can be as simple as the following:: config_host_data.set('HAVE_BTRFS_H', cc.has_header('linux/btrfs.h')) @@ -351,22 +377,30 @@ script, which may point to something other than the first python3 binary on the path. -Stage 3: makefiles -================== +Stage 3: Make +============= -The use of GNU make is required with the QEMU build system. +The next step in building QEMU is to invoke make. GNU Make is required +to build QEMU, and may be installed as ``gmake`` on some hosts. -The output of Meson is a build.ninja file, which is used with the Ninja -build system. QEMU uses a different approach, where Makefile rules are -synthesized from the build.ninja file. The main Makefile includes these -rules and wraps them so that e.g. submodules are built before QEMU. -The resulting build system is largely non-recursive in nature, in -contrast to common practices seen with automake. +The output of Meson is a ``build.ninja`` file, which is used with the +Ninja build tool. However, QEMU's build comprises other components than +just the emulators (namely firmware and the tests in ``tests/tcg``) which +need different cross compilers. The QEMU Makefile wraps both Ninja and +the smaller build systems for firmware and tests; it also takes care of +running ``configure`` again when the script changes. Apart from invoking +these sub-Makefiles, the resulting build is largely non-recursive. -Tests are also ran by the Makefile with the traditional ``make check`` -phony target, while benchmarks are run with ``make bench``. Meson test -suites such as ``unit`` can be ran with ``make check-unit`` too. It is also -possible to run tests defined in meson.build with ``meson test``. +Tests, whether defined in ``meson.build`` or not, are also ran by the +Makefile with the traditional ``make check`` phony target, while benchmarks +are run with ``make bench``. Meson test suites such as ``unit`` can be ran +with ``make check-unit``, and ``make check-tcg`` builds and runs "non-Meson" +tests for all targets. + +If desired, it is also possible to use ``ninja`` and ``meson test``, +respectively to build emulators and run tests defined in meson.build. +The main difference is that ``make`` needs the ``-jN`` flag in order to +enable parallel builds or tests. Useful make targets ------------------- @@ -378,6 +412,7 @@ Useful make targets Print the value of the variable VAR. Useful for debugging the build system. + Important files for the build system ==================================== @@ -391,8 +426,7 @@ number of dynamically created files listed later. ``Makefile`` The main entry point used when invoking make to build all the components of QEMU. The default 'all' target will naturally result in the build of - every component. Makefile takes care of recursively building submodules - directly via a non-recursive set of rules. + every component. ``*/meson.build`` The meson.build file in the root directory is the main entry point for the @@ -401,59 +435,92 @@ number of dynamically created files listed later. other meson.build files spread throughout the QEMU source tree. ``tests/Makefile.include`` - Rules for external test harnesses. These include the TCG tests, - ``qemu-iotests`` and the Avocado-based integration tests. + Rules for external test harnesses. These include the TCG tests + and the Avocado-based integration tests. ``tests/docker/Makefile.include`` - Rules for Docker tests. Like tests/Makefile, this file is included - directly by the top level Makefile, anything defined in this file will - influence the entire build system. + Rules for Docker tests. Like ``tests/Makefile.include``, this file is + included directly by the top level Makefile, anything defined in this + file will influence the entire build system. ``tests/vm/Makefile.include`` - Rules for VM-based tests. Like tests/Makefile, this file is included - directly by the top level Makefile, anything defined in this file will - influence the entire build system. + Rules for VM-based tests. Like ``tests/Makefile.include``, this file is + included directly by the top level Makefile, anything defined in this + file will influence the entire build system. Dynamically created files ------------------------- -The following files are generated dynamically by configure in order to -control the behaviour of the statically defined makefiles. This avoids -the need for QEMU makefiles to go through any pre-processing as seen -with autotools, where Makefile.am generates Makefile.in which generates -Makefile. +The following files are generated at run-time in order to control the +behaviour of the Makefiles. This avoids the need for QEMU makefiles to +go through any pre-processing as seen with autotools, where configure +generates ``Makefile`` from ``Makefile.in``. Built by configure: ``config-host.mak`` When configure has determined the characteristics of the build host it - will write a long list of variables to config-host.mak file. This - provides the various install directories, compiler / linker flags and a + will write them to this file for use in ``Makefile`` and to a smaller + extent ``meson.build``. These include the paths to various tools and a variety of ``CONFIG_*`` variables related to optionally enabled features. - This is imported by the top level Makefile and meson.build in order to - tailor the build output. - config-host.mak is also used as a dependency checking mechanism. If make + ``config-host.mak`` is also used as a dependency checking mechanism. If make sees that the modification timestamp on configure is newer than that on - config-host.mak, then configure will be re-run. + ``config-host.mak``, then configure will be re-run. - The variables defined here are those which are applicable to all QEMU - build outputs. Variables which are potentially different for each - emulator target are defined by the next file... + The variables defined here apply to all QEMU + build outputs. +``config-meson.cross`` + + A Meson "cross file" (or native file) used to communicate the paths to + the toolchain and other configuration options. + +``config.status`` + + A small shell script that will invoke configure again with the same + environment variables that were set during the first run. It's used to + rerun configure after changes to the source code, but it can also be + inspected manually to check the contents of the environment. + +``Makefile.prereqs`` + + A set of Makefile dependencies that order the build and execution of + firmware and tests after the container images and emulators that they + need. + +``pc-bios/*/config.mak``, ``tests/tcg/config-host.mak``, ``tests/tcg/*/config-target.mak`` + + Configuration variables used to build the firmware and TCG tests, + including paths to cross compilation toolchains. + +``pyvenv`` + + A Python virtual environment that is used for all Python code running + during the build. Using a virtual environment ensures that even code + that is run via ``sphinx-build``, ``meson`` etc. uses the same interpreter + and packages. Built by Meson: -``${TARGET-NAME}-config-devices.mak`` - TARGET-NAME is again the name of a system or userspace emulator. The - config-devices.mak file is automatically generated by make using the - scripts/make_device_config.sh program, feeding it the - default-configs/$TARGET-NAME file as input. +``config-host.h`` + Used by C code to determine the properties of the build environment + and the set of enabled features for the entire build. -``config-host.h``, ``$TARGET_NAME-config-target.h``, ``$TARGET_NAME-config-devices.h`` - These files are used by source code to determine what features are - enabled. They are generated from the contents of the corresponding - ``*.mak`` files using Meson's ``configure_file()`` function. +``${TARGET-NAME}-config-devices.mak`` + TARGET-NAME is the name of a system emulator. The file is + generated by Meson using files under ``configs/devices`` as input. + +``${TARGET-NAME}-config-target.mak`` + TARGET-NAME is the name of a system or usermode emulator. The file is + generated by Meson using files under ``configs/targets`` as input. + +``$TARGET_NAME-config-target.h``, ``$TARGET_NAME-config-devices.h`` + Used by C code to determine the properties and enabled + features for each target. enabled. They are generated from + the contents of the corresponding ``*.mak`` files using Meson's + ``configure_file()`` function; each target can include them using + the ``CONFIG_TARGET`` and ``CONFIG_DEVICES`` macro respectively. ``build.ninja`` The build rules.