From patchwork Mon Dec 8 09:08:12 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ernestas Kulik X-Patchwork-Id: 2172494 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=googlegroups.com header.i=@googlegroups.com header.a=rsa-sha256 header.s=20230601 header.b=BnkceA8q; dkim-atps=neutral Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=googlegroups.com (client-ip=2a00:1450:4864:20::13e; helo=mail-lf1-x13e.google.com; envelope-from=swupdate+bncbc33pig2rudrbgnn3leqmgqekhzrrly@googlegroups.com; receiver=patchwork.ozlabs.org) Received: from mail-lf1-x13e.google.com (mail-lf1-x13e.google.com [IPv6:2a00:1450:4864:20::13e]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange x25519 server-signature ECDSA (secp384r1) server-digest SHA384) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4dPx8w14w5z1xvn for ; Mon, 08 Dec 2025 20:13:03 +1100 (AEDT) Received: by mail-lf1-x13e.google.com with SMTP id 2adb3069b0e04-5958fe96529sf2562265e87.3 for ; Mon, 08 Dec 2025 01:13:03 -0800 (PST) ARC-Seal: i=2; a=rsa-sha256; t=1765185180; cv=pass; d=google.com; s=arc-20240605; b=fapoxIlCQjBO4PqcZBkNBPUZvR291nR+NK+w73DimVeLfg7rxoSGExGbkpX9WIvo8b fNGBctNnG21RQEtL6U32tSpLomMGcOiqWPi5Itlke69BWB7Dl+o6dFOmYg4pER0d2yLS yDy8o1U/Y+V17rUU8WaZjPDs3fULO8q/vablYZ81LvtfqTC5EDCVp6EdWXqmb0w/vNi0 Hyw2vaNV5m+pmy6/9j8gq6h6TvQjWEcGQsVXWXveAvJTUB6t0o2/A5exjkWIkInyNBlF hn2CjO357KXtVU47JRjBFhwCRfPvvtaRNmPdVnCh4+owwacr4XW8IUw9Ad788sqPeWWN k6rw== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20240605; h=list-unsubscribe:list-subscribe:list-archive:list-help:list-post :list-id:mailing-list:precedence:reply-to:mime-version:references :in-reply-to:message-id:date:subject:cc:to:from:dkim-signature; bh=0Ec5lNtOHY4I7GTslfWaMKdL13t1SxocpbQiQwO4c4w=; fh=+YQfhfL+sgrWi1eBD28PyKHyrmjqMDpTVHnWzNE3fTo=; b=fn9JvElaUqSK/N0zalZJO/0z2r2uBUMZF1NXQj3uQLg4Frb8ZiEqGvdPCFpLBnKBZ3 6r9RWfG4uisz45+sR7vRwKnaQeGQtt+BOXMGle+6fNxKuJsK8/uq6I3f/F5pPTL++ndz l8Qkduayjan+aJgVqYUKoyNsvRwhAKj4JTE40ulYUy4Qx6t6fZU+huNxTl3wTsbOHcQ+ GHpaXcT7ZVDQsSF6Blf40J+tipKCilQrCZkgXxwd+MT5N0cMJOveHvDjYLMPjbQIBnmy eiMPJPhzOe9fMsuzNG+1hfBudwMjic+bc/ftR8T9KBdcKQgE+3YOAkYzBeTPUeS/48+6 iTRg==; darn=patchwork.ozlabs.org ARC-Authentication-Results: i=2; gmr-mx.google.com; dkim=pass header.i=@iconn-networks.com header.s=default header.b=h5qA0g87; spf=pass (google.com: domain of ernestas.k@iconn-networks.com designates 62.77.155.212 as permitted sender) smtp.mailfrom=ernestas.k@iconn-networks.com; dmarc=pass (p=QUARANTINE sp=QUARANTINE dis=NONE) header.from=iconn-networks.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=googlegroups.com; s=20230601; t=1765185180; x=1765789980; darn=patchwork.ozlabs.org; h=list-unsubscribe:list-subscribe:list-archive:list-help:list-post :list-id:mailing-list:precedence:reply-to :x-original-authentication-results:x-original-sender:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from:from:to :cc:subject:date:message-id:reply-to; bh=0Ec5lNtOHY4I7GTslfWaMKdL13t1SxocpbQiQwO4c4w=; b=BnkceA8qsdjxV1M25Pi3Sb2FBIiWLbX941rNvFwmv8LwjQRdt1GvtImOFkisKhXpMV KlkWjGd0W4bB8X9pUySzjrfTmpcX4xVqDn5ixWoxszJVvRpK7o9FlJwtZ1YWCmQEL38q 9FeXBrH+64/3bHOvjlSdCX7DzRApGLozH0Xrjbs6CVuy5CKIHR1m1jT3Z/cL8LdgQyhG b3/uoHcYwHUwJ3qBzTHv/BYLxp8NETA462rLUG0n2YYowAZEYvqw/arAdApHZrQ/lrNd mIJXqg8CKWLunQX+XW2KTDW3sPb2GIUJJKBb47/8YoXea8Iqy9pFW/RlPtDEZ9y2dT3Y HaVg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1765185180; x=1765789980; h=list-unsubscribe:list-subscribe:list-archive:list-help:list-post :x-spam-checked-in-group:list-id:mailing-list:precedence:reply-to :x-original-authentication-results:x-original-sender:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :x-beenthere:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=0Ec5lNtOHY4I7GTslfWaMKdL13t1SxocpbQiQwO4c4w=; b=gt1PT09X+fBX5h1tD2nQpTe2KouspHgRt+wvZSRKrZpdNp7eD+4MwKRC6VvyC0YxDO IU9XKpbJfxzGwLSCrBTwuQEZ6XSecEgiPVZKe9ZU+4+fcRJnefgMbhMir0EYQ3tZbh1f LcD1U9A1DIa5L9LLbZ8Dj5P4bpLz1k9AxEOX79rr88RM3gto8aJsEBcpwUN43q2YQapU x6uLnxwAXZm7AslPqR+/d8TSIAJBJLeCqp0320kHNDUreZ6bKfw4VN22ohHY84MGyWfi bSEGi/a41uc8UTtsms3sIO3mLMJsHpccCPVs1aBAl1Ck4iiCLucMYDfKUCmmdH8i3Jxj Itaw== X-Forwarded-Encrypted: i=2; AJvYcCWTGMNga3gNbZQkUZBHq9ssmFJv/E2bdwG5dBaeAbwcbh20x9XJo+vo9DFXkcH4Qs6FyvGTaplePQ==@patchwork.ozlabs.org X-Gm-Message-State: AOJu0YxDP/0idtQu0pjqC4mp2DW/0Xm2iwTQ6OyujBI4G5h169tIH3x7 KTSP+DGwAoXdwxtpQXbFlY4GvKPSMqXJt0qtd80U9WTra1SJ4/FEwRu2 X-Google-Smtp-Source: AGHT+IE9sgrG4+XDfqu0LHYNSzpXZ5RMiDLpvg6QAsLpwIceSB+2/ms5ofz8bCzaKXJlKDMEFrlEWA== X-Received: by 2002:a05:6512:3ba3:b0:595:7e4e:45d7 with SMTP id 2adb3069b0e04-5987e88ba7dmr2109788e87.12.1765185180329; Mon, 08 Dec 2025 01:13:00 -0800 (PST) X-BeenThere: swupdate@googlegroups.com; h="AWVwgWaOQosp4yW54SklSbiE9G1eOGDGXtYberHJcoEoGWQWrA==" Received: by 2002:a05:6512:3416:b0:598:e79c:3836 with SMTP id 2adb3069b0e04-598e79c3ac5ls60423e87.1.-pod-prod-04-eu; Mon, 08 Dec 2025 01:12:55 -0800 (PST) X-Received: by 2002:a05:6512:3985:b0:594:282c:ba6b with SMTP id 2adb3069b0e04-598853b23e5mr1864912e87.28.1765185175404; Mon, 08 Dec 2025 01:12:55 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1765185175; cv=none; d=google.com; s=arc-20240605; b=TL8UciYRInlUqyE9Rp/3OgBBWzQlSnOocFyLuRf3x45nsxO798KIEO3R6p/Tz2Rc2W F9zjaP/r2qK9hS2M0voBxHsZhq9CzBCk8CSZpxMl7zQuJSwgrfrYAj8nuiq5v/7q6kfL wO833fd2oo1xZs64FsbAhVU4ELgwo6qC3deRbPuZW8SlBUqKjSlPNamvqtVP2D3UIMI+ yfpE8Xq5Jz3lC1PO9w2HBnonxkSXQt8H8menhDgegrmc95mXeMcEYVMpMM8ecJYwSrgZ 4QfoYkInhIcYISj1xZGJ9S8eW13xlGv9Blqcp3TY40iVSHlEd1olNzFLWBnxvaRAcirN sh/w== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20240605; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:dkim-signature; bh=NJeNp5UmbVafr4uWTPVp5TUMdL+IOIlnpQqOaSTaSIk=; fh=z5EU8Jpv4PD/2kyu/5TapovCO0ItxGnnOw5ONyN7/gQ=; b=XY3aAUSYlJohsqT2zgT1wPdPIyAIAp3yNfTSD9WZfFbc9RA4cpQOpb4lnoSlgGyW+w fMoQruaSxF97jh1BJne70Va9MS3npEdbJwGRc3cELAGmW5dNq01odpoBfoglKcDTpkc6 watmI57CSvJiwV0nhJKMYVLn14cEDFnsaXR9mmULVwdY9KETidoode5ENw/1NXlhB/R1 CXIBZKYQLiNxGjJbqQoecmKYpw8jxjSU6jUbufiXLiG6odmw/hwJuBDXScYzWyJ0enyI Sv+NP95rdHQzQg+6LDXyjnJruqAZlBbWnkrWtR05JjOqhw5cpa9jNvhdivwIqvyPdmSt NlIg==; dara=google.com ARC-Authentication-Results: i=1; gmr-mx.google.com; dkim=pass header.i=@iconn-networks.com header.s=default header.b=h5qA0g87; spf=pass (google.com: domain of ernestas.k@iconn-networks.com designates 62.77.155.212 as permitted sender) smtp.mailfrom=ernestas.k@iconn-networks.com; dmarc=pass (p=QUARANTINE sp=QUARANTINE dis=NONE) header.from=iconn-networks.com Received: from zebra.wilibox.com (zebra.wilibox.com. [62.77.155.212]) by gmr-mx.google.com with ESMTPS id 2adb3069b0e04-597d7b1a3cesi145294e87.1.2025.12.08.01.12.55 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 08 Dec 2025 01:12:55 -0800 (PST) Received-SPF: pass (google.com: domain of ernestas.k@iconn-networks.com designates 62.77.155.212 as permitted sender) client-ip=62.77.155.212; Received: from debian.. (unknown [81.7.79.211]) (Authenticated sender: ernestas.k@iconn-networks.com) by zebra.wilibox.com (Postfix) with ESMTPSA id 65619172E73; Mon, 8 Dec 2025 11:12:20 +0200 (EET) X-Patchwork-Original-From: "'Ernestas Kulik' via swupdate" From: Ernestas Kulik To: swupdate@googlegroups.com Cc: toertel@gmail.com, Ernestas Kulik Subject: [swupdate] [swugenerator] [PATCH v3 1/2] generator: Implement xz support Date: Mon, 8 Dec 2025 11:08:12 +0200 Message-ID: <20251208091154.2027780-2-ernestas.k@iconn-networks.com> X-Mailer: git-send-email 2.47.3 In-Reply-To: <20251208091154.2027780-1-ernestas.k@iconn-networks.com> References: <20251208091154.2027780-1-ernestas.k@iconn-networks.com> MIME-Version: 1.0 X-Original-Sender: ernestas.k@iconn-networks.com X-Original-Authentication-Results: gmr-mx.google.com; dkim=pass header.i=@iconn-networks.com header.s=default header.b=h5qA0g87; spf=pass (google.com: domain of ernestas.k@iconn-networks.com designates 62.77.155.212 as permitted sender) smtp.mailfrom=ernestas.k@iconn-networks.com; dmarc=pass (p=QUARANTINE sp=QUARANTINE dis=NONE) header.from=iconn-networks.com X-Original-From: Ernestas Kulik Reply-To: Ernestas Kulik Precedence: list Mailing-list: list swupdate@googlegroups.com; contact swupdate+owners@googlegroups.com List-ID: X-Spam-Checked-In-Group: swupdate@googlegroups.com X-Google-Group-Id: 605343134186 List-Post: , List-Help: , List-Archive: , List-Unsubscribe: , While swupdate is capable of decompressing XZ archives, swugenerator currently does not allow its use in compression. This commit adds support for it and refactors the compressed entry processing for maintainability. Signed-off-by: Ernestas Kulik Reviewed-by: Mark Jonas --- swugenerator/generator.py | 78 ++++++++++++++++----------------------- 1 file changed, 32 insertions(+), 46 deletions(-) diff --git a/swugenerator/generator.py b/swugenerator/generator.py index 4a4ce8e..706b850 100644 --- a/swugenerator/generator.py +++ b/swugenerator/generator.py @@ -64,10 +64,40 @@ class SWUGenerator: def close(self): self.temp.cleanup() self.cpiofile.add_trailer() self.out.close() + def process_compressed_entry(self, entry, cmp, new): + cmds = { + "xz": ["xz", "-f", "-k", "-c"], + "zlib": ["gzip", "-f", "-9", "-n", "-c", "--rsyncable"], + "zstd": ["zstd", "-z", "-k", "-T0", "-f", "-c"], + } + cmd = cmds.get(cmp) + if not cmd: + logging.critical("Wrong compression algorithm: %s", cmp) + sys.exit(1) + + new_path = os.path.join(self.temp.name, new.newfilename) + "." + cmp + new.newfilename = new.newfilename + "." + cmp + + cmd.extend([new.fullfilename, ">", new_path]) + + try: + subprocess.run(" ".join(cmd), shell=True, check=True, text=True) + except subprocess.CalledProcessError: + logging.critical( + "Cannot compress %s with %s", entry["filename"], cmd + ) + sys.exit(1) + + new.fullfilename = new_path + + if entry.get("type") == "ubivol": + entry.setdefault("properties", {}) \ + .update({ "decompressed-size": str(new.getsize()) }) + def process_entry(self, entry): if "filename" not in entry: return new = None for image in self.artifacts: @@ -81,56 +111,12 @@ class SWUGenerator: logging.critical("Artifact %s not found", entry["filename"]) sys.exit(22) new.newfilename = entry["filename"] - if "compressed" in entry and not self.nocompress: - cmp = entry["compressed"] - if cmp not in ("zlib", "zstd"): - logging.critical("Wrong compression algorithm: %s", cmp) - sys.exit(1) - - new_path = os.path.join(self.temp.name, new.newfilename) + "." + cmp - new.newfilename = new.newfilename + "." + cmp - if cmp == "zlib": - cmd = [ - "gzip", - "-f", - "-9", - "-n", - "-c", - "--rsyncable", - new.fullfilename, - ">", - new_path, - ] - else: - cmd = [ - "zstd", - "-z", - "-k", - "-T0", - "-f", - "-c", - new.fullfilename, - ">", - new_path, - ] - - try: - subprocess.run(" ".join(cmd), shell=True, check=True, text=True) - except subprocess.CalledProcessError: - logging.critical( - "Cannot compress %s with %s", entry["filename"], cmd - ) - sys.exit(1) - - new.fullfilename = new_path - - if entry.get("type") == "ubivol": - entry.setdefault("properties", {}) \ - .update({ "decompressed-size": str(new.getsize()) }) + if not self.nocompress and (cmp := entry.get("compressed")): + self.process_compressed_entry(entry, cmp, new) # compression cannot be used with delta, because it has own compressor elif ("type" in entry) and entry["type"] == "delta": cmd = [ "zck", "-u",