From patchwork Sun Dec 1 21:13:04 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dan Walkes X-Patchwork-Id: 2016900 X-Patchwork-Delegate: sbabic@denx.de Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=googlegroups.com header.i=@googlegroups.com header.a=rsa-sha256 header.s=20230601 header.b=GrECrtWC; dkim-atps=neutral Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=googlegroups.com (client-ip=2607:f8b0:4864:20::837; helo=mail-qt1-x837.google.com; envelope-from=swupdate+bncbcwltgh224mbbzvcwo5amgqew3jssoi@googlegroups.com; receiver=patchwork.ozlabs.org) Received: from mail-qt1-x837.google.com (mail-qt1-x837.google.com [IPv6:2607:f8b0:4864:20::837]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (secp384r1) server-digest SHA384) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4Y1fld1fj1z1yQN for ; Mon, 2 Dec 2024 08:13:15 +1100 (AEDT) Received: by mail-qt1-x837.google.com with SMTP id d75a77b69052e-4668f208ff7sf61960091cf.0 for ; Sun, 01 Dec 2024 13:13:15 -0800 (PST) ARC-Seal: i=2; a=rsa-sha256; t=1733087592; cv=pass; d=google.com; s=arc-20240605; b=JI+6KmU2ip9rT/+zXFKXYqtDEjUSMV58e9f5cPq9MECfBAUi6eWBhuDMiDBzCGVcNi V5ejRoQtSd/dHt8jSW69TacXp5NqGUqC33QCuqc+PalmPkzY71WuKmvjFkp9GpjMu0y8 JZ4gh3Xjeku25pBWrtYFG/inWfILVslYGpH7svHwWaM21rWWz7ZepCt9YAd0ACjr/6Fk puRglct1xNFUBS9ymlOHCJtS3dKj1RwIbPi+8PDniWP0GhdObqXvW1wqP9KIB+BhwkpI md5UIrlh2J/aox0aAHwzuJLDPyfpUretctkYgoiYdAI1PRm8lgJwIBJ1x9zaBnfE7yk+ 5bCw== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20240605; h=list-unsubscribe:list-subscribe:list-archive:list-help:list-post :list-id:mailing-list:precedence:mime-version:message-id:date :subject:cc:to:from:sender:dkim-signature; bh=4dzDPCAOVtbzuSghJi5jxEX0n+IAzzsxSuUhRG8tOa8=; fh=szge2Lp01TMdJa1nWnrzzWzCH7R0r6UdckxPkSAkvpM=; b=bxg708n2YIEy329yCF7XrAWWFWvb7Af967rN3IQRq1zEHmu/6ZjX4aoBS9ZC0z/eHK 8o+KOiv+ZT22x96v0no6quLfYCKXqTVjh1MSCldPO9+j1l3OJ9s8KuvLEuHDKt26AmdQ RG3jmt2tsXMIIHIzZvtkfDxSfj3n6+00NyBJOAaAMTPPVuvYwI3VPlyEsyhFt5sg+4Tv bm04L8ul1Cjt0iAvhWW4vQs5NvlMkSJ6UfqfQ7ow8Byy1udnwpwy/wvRTsoZBZfTToAv Es5d+qZKIPR9gBglanLhIz33eqjdFQPXW8yjrBqFEXvzou/0daO45nEN7eWLheCEJdgh Kiig==; darn=patchwork.ozlabs.org ARC-Authentication-Results: i=2; gmr-mx.google.com; dkim=pass header.i=@trellis-logic-com.20230601.gappssmtp.com header.s=20230601 header.b=jxW8kKcr; spf=none (google.com: danwalkes@trellis-logic.com does not designate permitted sender hosts) smtp.mailfrom=danwalkes@trellis-logic.com; dara=pass header.i=@googlegroups.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=googlegroups.com; s=20230601; t=1733087592; x=1733692392; darn=patchwork.ozlabs.org; h=list-unsubscribe:list-subscribe:list-archive:list-help:list-post :list-id:mailing-list:precedence:x-original-authentication-results :x-original-sender:mime-version:message-id:date:subject:cc:to:from :sender:from:to:cc:subject:date:message-id:reply-to; bh=4dzDPCAOVtbzuSghJi5jxEX0n+IAzzsxSuUhRG8tOa8=; b=GrECrtWCPS3Bvbokt5huOlr5oYqcJ3lE5rc435/SS2GYwjuLeb1pzHa5WC5EuLpsxz OJsareyTwDJicm5F3O52YwX6imsK8IiYmsBnQ817zewbnHr4+J6hLT3CHOtMIq/kDZXx jWKlqFhi2+2hohXDGS9xiwiCeBGhIS5lZ0gQvscQkfixJkBUB6SgOWMiynrZtL3EKQem L9zgRn4WuIDs7+4jnNE8rFrEXluHH6dP4comDTQLEHBs7Vx9FTTNMKAVpOw3OQUdgABp 5S52rpCz4LYTG9Rd0byLyS1Haiiot0yCOLn8QhmsfR3waZxdjkpDhf7uFKVxtGQCk7kn DX6Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1733087592; x=1733692392; h=list-unsubscribe:list-subscribe:list-archive:list-help:list-post :x-spam-checked-in-group:list-id:mailing-list:precedence :x-original-authentication-results:x-original-sender:mime-version :message-id:date:subject:cc:to:from:x-beenthere:x-gm-message-state :sender:from:to:cc:subject:date:message-id:reply-to; bh=4dzDPCAOVtbzuSghJi5jxEX0n+IAzzsxSuUhRG8tOa8=; b=obeqQ/tXfylPr/kulpoFFDLaw/DOryq8F+4y31UKh/+/GHTL1KwM1Y0BCIvKiyeIoK ih8zRcvnRNw8Ft8Mu8T0Tn3oY5KF/7QxzmCnKnJQXqH22xUMAKnWbobsCS6knr/PaeQi gUH5179nUhSqnH23ZU56h+aj/QAUWA/cLBE9QwyyvQmuFHrE63hAMq9GhzouaOd/RSab MhLM9VFuYqUTxUqbF0a/zG8ozQr6AQ0PlPsutYx5OW+MDE3I8G3nM30sp7p6redMrf4s D0gZPM2wy1TNBiMpYAyRrW5r1XtavAKduTEQ97HKB0CHmvI1YCJtJ5oMs2NuTESaFxmB 3BnA== Sender: swupdate@googlegroups.com X-Forwarded-Encrypted: i=2; AJvYcCV/qD9Htxkp/wc8TgewTjOqJP2KhvjLgB8tcLIaOa1XhrwzqQKKuuRObazWHHsH+PrlJwmxXoOpKQ==@patchwork.ozlabs.org X-Gm-Message-State: AOJu0YwnX1FTr2dbdtQ5D4AHaKoNphvKV5r5ecPttuA/UoMkBtZV+NR3 884tAjdd5wVw++SATnBWnYsrbqxTAnQKni29GOInxItn2AT8/Ghk X-Google-Smtp-Source: AGHT+IFs48eYgAgBqNus7WenxMamQoOuUDh+mSlsxRHXJz3Xi9KlffGPM3iqVsqVVfLSQa744ObxuA== X-Received: by 2002:a05:622a:2ca:b0:461:7262:4153 with SMTP id d75a77b69052e-466c1b2360fmr295158651cf.10.1733087591785; Sun, 01 Dec 2024 13:13:11 -0800 (PST) X-BeenThere: swupdate@googlegroups.com Received: by 2002:ac8:580e:0:b0:458:3f09:c712 with SMTP id d75a77b69052e-466c1d20c4fls20054151cf.0.-pod-prod-00-us; Sun, 01 Dec 2024 13:13:10 -0800 (PST) X-Received: by 2002:a05:622a:1825:b0:466:93b9:8356 with SMTP id d75a77b69052e-466c1b9e229mr281964561cf.22.1733087589563; Sun, 01 Dec 2024 13:13:09 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1733087589; cv=none; d=google.com; s=arc-20240605; b=G8dRX8ui4kuuzh3sdrxOEdrxacBWYmrZrQ6VvBBp5O5f/POXsM4jEqnmQzlob9PWkx ENX6jfoVZxgz0s8RfU850o0wHa/4qx6BTZQjtEXFZjYMER3BQpezVdSwlUJnf45jklXP HBQPSkAlj6V/S5Usf5E21F9nfHjuIUW/cMb/Lu4S2vy+ggFpq143AbgkMQ7E78TS9EFX OcZhy3FNQ9VIwBHc66GC0+JIIgX8/xOBdV3SUL+axRRKO/Nw9BmHIYx4zZb2lGle3lSU RbF3yiyqd3kREOzfEaHFpG4z1aXxlLSWe02A2Eo4IpwvA0R4caFx09Zutth6GsALAnoi fJog== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20240605; h=content-transfer-encoding:mime-version:message-id:date:subject:cc :to:from:dkim-signature; bh=Fbquef2YYPfLNprJt/kg5v3ACGJp6QK1fLilKOZWVp4=; fh=CPPy5SrA2Xeg3agvV+7qgllTb4Osr3SZWuzMNAf7F0A=; b=HQiB3otb/aR4fKhbLk53egxGysfYDKTFTOGsll2W8HAl3IsVBF+xavOlM6tzvN6WFV nmqoOTRKJ8M+ep5QKMy5M3vzrkki67NFREWKAvG53fzNgDdu1mkOVFiYnWcJlRRGUbG1 0i++AkMjq67mJo3+iAfruzLkMSPRliTdTVM/cbeUbyUkw7LbuUfq0IbcuXct5hpti1Ts xOgdfFgOAPMqKkjb0gR9NDLJneN842gAA5O9RQZ2ZO8bJgDmgVWUvdfoumsCetz506Gi XiM0zo3RdfS0qbc206/UpVoWEDu9f0vAZ2dZ5ALobrxQbFtkJaK48l3NQ4bnjIDWqd7C TLNA==; dara=google.com ARC-Authentication-Results: i=1; gmr-mx.google.com; dkim=pass header.i=@trellis-logic-com.20230601.gappssmtp.com header.s=20230601 header.b=jxW8kKcr; spf=none (google.com: danwalkes@trellis-logic.com does not designate permitted sender hosts) smtp.mailfrom=danwalkes@trellis-logic.com; dara=pass header.i=@googlegroups.com Received: from mail-io1-xd2d.google.com (mail-io1-xd2d.google.com. [2607:f8b0:4864:20::d2d]) by gmr-mx.google.com with ESMTPS id d75a77b69052e-466c4230ec0si3316811cf.4.2024.12.01.13.13.09 for (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Sun, 01 Dec 2024 13:13:09 -0800 (PST) Received-SPF: none (google.com: danwalkes@trellis-logic.com does not designate permitted sender hosts) client-ip=2607:f8b0:4864:20::d2d; Received: by mail-io1-xd2d.google.com with SMTP id ca18e2360f4ac-84198252db1so98767439f.0 for ; Sun, 01 Dec 2024 13:13:09 -0800 (PST) X-Gm-Gg: ASbGncv53jy+nPCpfoFRNVGY3zt7IFQnMvVFAxXKTSpJ6Nr02ds9270PJrc9Vrkwmv2 JrBPKgUuqRpdb3HHuEOYjze4QsRXEWFSRsREVq6J0HcHj3AjzrJK3TsT8A+iwDzsgRLSbChB1m0 bHzl3vtdm5/caOwUb9rtZU1LH5KoNFlfLbdbvVhpVf5Z9+8h8tqlQ68to6V7WZhkL6GBfVoYnoy Q/DZfh97TTCNlzg6BaaMW7u9y/6ro7pMvEGcFaxvXML8rRmZ3DL4pJ2R4ZrvTBom9T4cAPed9de NjmD44xusbU+Lh7a/Q== X-Received: by 2002:a5e:a60d:0:b0:841:9225:1f56 with SMTP id ca18e2360f4ac-84401589d4amr1286432939f.3.1733087588539; Sun, 01 Dec 2024 13:13:08 -0800 (PST) Received: from dan-tr.. (c-98-245-195-12.hsd1.co.comcast.net. [98.245.195.12]) by smtp.gmail.com with ESMTPSA id 8926c6da1cb9f-4e230da8d86sm1749891173.20.2024.12.01.13.13.07 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 01 Dec 2024 13:13:07 -0800 (PST) From: Dan Walkes To: swupdate@googlegroups.com Cc: Dan Walkes Subject: [swupdate] [meta-swupdate][PATCH] swupdate classes: fix hash mismatch on master Date: Sun, 1 Dec 2024 14:13:04 -0700 Message-Id: <20241201211304.2199299-1-danwalkes@trellis-logic.com> X-Mailer: git-send-email 2.34.1 MIME-Version: 1.0 X-Original-Sender: danwalkes@trellis-logic.com X-Original-Authentication-Results: gmr-mx.google.com; dkim=pass header.i=@trellis-logic-com.20230601.gappssmtp.com header.s=20230601 header.b=jxW8kKcr; spf=none (google.com: danwalkes@trellis-logic.com does not designate permitted sender hosts) smtp.mailfrom=danwalkes@trellis-logic.com; dara=pass header.i=@googlegroups.com Precedence: list Mailing-list: list swupdate@googlegroups.com; contact swupdate+owners@googlegroups.com List-ID: X-Spam-Checked-In-Group: swupdate@googlegroups.com X-Google-Group-Id: 605343134186 List-Post: , List-Help: , List-Archive: , List-Unsubscribe: , The fix discussed in [1] resolves build issues on styhead but introduces an issue which causes hash mismatches. The first build succeeds to generate a working .swu, however future builds fail with hash mismatch and a message like this: ``` [ERROR] : SWUPDATE failed [0] ERROR cpio_utils.c : __swupdate_copy : 670 : HASH mismatch : 5bb39965bfcedd87fc02bd51fac6636b924aed606d0984dcebae6a3250b01af6 <--> ff70a8dbc50a9ce2bad724554f5f14b8c933ad87fc544d054142755e8ef07f1e [ERROR] : SWUPDATE failed [0] ERROR archive_handler.c : install_archive_image : 344 : Error copying extracted file ``` This happens because the modifications made to files in the S directory only happen once. This means there's a single conversion of `$swupdate_get_sha256(` to the file sha, and future builds use the hash previously placed in the file in the S directory which is no longer correct when the rootfs file hash changes. It would probably be possible to make a simpler change here which splits the UNPACKDIR and S and continues to make runtime edits in S. However, it seems the intent of S and/or UNPACKDIR is to contain unmodified source as discussed in [1]. This change instead: 1. Moves the sw-description file to WORKDIR for editing. 2. Continues to use S as source location for sw-description and other source files which are not edited. 3. Makes all edits in WORKDIR. 4. When preparing the cpio, prepares from WORKDIR instead of S. using any existing files in WORKDIR if they exist. If they do not exist, symlinks the relevant files in S instead. 6. Adds the -L option to cpio to prevent adding symlinks to the cpio file, and avoid the need to copy from S to WORKDIR when preparing the cpio. 1: https://groups.google.com/g/swupdate/c/8K-9H7C9o5E/m/a9fqhIOjAAAJ Signed-off-by: Dan Walkes --- classes-recipe/swupdate-common.bbclass | 37 +++++++++++++++----------- classes-recipe/swupdate-image.bbclass | 2 +- 2 files changed, 23 insertions(+), 16 deletions(-) diff --git a/classes-recipe/swupdate-common.bbclass b/classes-recipe/swupdate-common.bbclass index 7b49561..085a7b5 100644 --- a/classes-recipe/swupdate-common.bbclass +++ b/classes-recipe/swupdate-common.bbclass @@ -66,22 +66,22 @@ def swupdate_getdepends(d): return depstr -def swupdate_write_sha256(s): +def swupdate_write_sha256(workdir): import re write_lines = [] - with open(os.path.join(s, "sw-description"), 'r') as f: + with open(os.path.join(workdir, "sw-description"), 'r') as f: for line in f: shastr = r"sha256.+=.+@(.+\")" m = re.match(r"^(?P.+)(sha256|version).+[=:].*(?P[\'\"])@(?P.*)(?P=quote)", line) if m: filename = m.group('filename') bb.warn("Syntax for sha256 changed, please use $swupdate_get_sha256(%s)" % filename) - hash = swupdate_get_sha256(None, s, filename) + hash = swupdate_get_sha256(None, workdir, filename) write_lines.append(line.replace("@%s" % (filename), hash)) else: write_lines.append(line) - with open(os.path.join(s, "sw-description"), 'w+') as f: + with open(os.path.join(workdir, "sw-description"), 'w+') as f: for line in write_lines: f.write(line) @@ -97,7 +97,7 @@ def swupdate_exec_functions(d, s, write_lines): write_lines[index] = line -def swupdate_expand_bitbake_variables(d, s): +def swupdate_expand_bitbake_variables(d, s, workdir): write_lines = [] with open(os.path.join(s, "sw-description"), 'r') as f: @@ -131,7 +131,7 @@ def swupdate_expand_bitbake_variables(d, s): swupdate_exec_functions(d, s, write_lines) - with open(os.path.join(s, "sw-description"), 'w+') as f: + with open(os.path.join(workdir, "sw-description"), 'w+') as f: for line in write_lines: f.write(line) @@ -173,17 +173,18 @@ def prepare_sw_description(d): import subprocess s = d.getVar('S') - swupdate_expand_bitbake_variables(d, s) + workdir = d.getVar('WORKDIR') + swupdate_expand_bitbake_variables(d, s, workdir) - swupdate_write_sha256(s) + swupdate_write_sha256(workdir) encrypt = d.getVar('SWUPDATE_ENCRYPT_SWDESC') if encrypt: bb.note("Encryption of sw-description") - shutil.copyfile(os.path.join(s, 'sw-description'), os.path.join(s, 'sw-description.plain')) + shutil.copyfile(os.path.join(workdir, 'sw-description'), os.path.join(workdir, 'sw-description.plain')) key,iv = swupdate_extract_keys(d.getVar('SWUPDATE_AES_FILE')) - iv = swupdate_get_IV(d, s, 'sw-description') - swupdate_encrypt_file(os.path.join(s, 'sw-description.plain'), os.path.join(s, 'sw-description'), key, iv) + iv = swupdate_get_IV(d, workdir, 'sw-description') + swupdate_encrypt_file(os.path.join(workdir, 'sw-description.plain'), os.path.join(workdir, 'sw-description'), key, iv) signing = d.getVar('SWUPDATE_SIGNING') if signing == "1": @@ -191,8 +192,8 @@ def prepare_sw_description(d): signing = "RSA" if signing: - sw_desc_sig = os.path.join(s, 'sw-description.sig') - sw_desc = os.path.join(s, 'sw-description.plain' if encrypt else 'sw-description') + sw_desc_sig = os.path.join(workdir, 'sw-description.sig') + sw_desc = os.path.join(workdir, 'sw-description.plain' if encrypt else 'sw-description') if signing == "CUSTOM": signcmd = [] @@ -233,6 +234,7 @@ def swupdate_add_src_uri(d, list_for_cpio): import shutil s = d.getVar('S') + workdir = d.getVar('WORKDIR') exclude = (d.getVar("SWUPDATE_SRC_URI_EXCLUDE") or "").split() fetch = bb.fetch2.Fetch([], d) @@ -310,9 +312,14 @@ def swupdate_add_artifacts(d, list_for_cpio): def swupdate_create_cpio(d, swudeploydir, list_for_cpio): s = d.getVar('S') - os.chdir(s) + workdir = d.getVar('WORKDIR') + os.chdir(workdir) + for file in list_for_cpio: + if not os.path.exists(file): + os.symlink(os.path.join(s, file), file) + updateimage = d.getVar('IMAGE_NAME') + '.swu' - line = 'for i in ' + ' '.join(list_for_cpio) + '; do echo $i;done | cpio -ov -H crc --reproducible > ' + os.path.join(swudeploydir, updateimage) + line = 'for i in ' + ' '.join(list_for_cpio) + '; do echo $i;done | cpio -ov -H crc --reproducible -L > ' + os.path.join(swudeploydir, updateimage) os.system(line) os.chdir(swudeploydir) updateimage_link = d.getVar('IMAGE_LINK_NAME') diff --git a/classes-recipe/swupdate-image.bbclass b/classes-recipe/swupdate-image.bbclass index 1cd3eeb..bf8c5cc 100644 --- a/classes-recipe/swupdate-image.bbclass +++ b/classes-recipe/swupdate-image.bbclass @@ -40,7 +40,7 @@ python do_swupdate_copy_swdescription() { import shutil - workdir = d.getVar('S') + workdir = d.getVar('WORKDIR') filespath = d.getVar('FILESPATH') sw_desc_path = bb.utils.which(filespath, "sw-description") shutil.copyfile(sw_desc_path, os.path.join(workdir, "sw-description"))