From patchwork Wed Nov 24 12:30:26 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Stefano Babic X-Patchwork-Id: 1559068 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: bilbo.ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=googlegroups.com header.i=@googlegroups.com header.a=rsa-sha256 header.s=20210112 header.b=BaKJalnx; dkim-atps=neutral Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=googlegroups.com (client-ip=2a00:1450:4864:20::23a; helo=mail-lj1-x23a.google.com; envelope-from=swupdate+bncbcxploxj6ikrb5ha7cgamgqemaqrioy@googlegroups.com; receiver=) Received: from mail-lj1-x23a.google.com (mail-lj1-x23a.google.com [IPv6:2a00:1450:4864:20::23a]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by bilbo.ozlabs.org (Postfix) with ESMTPS id 4HzgMy1gY7z9ssD for ; Wed, 24 Nov 2021 23:30:52 +1100 (AEDT) Received: by mail-lj1-x23a.google.com with SMTP id o15-20020a2e90cf000000b00218dfebebdesf799966ljg.13 for ; Wed, 24 Nov 2021 04:30:52 -0800 (PST) ARC-Seal: i=2; a=rsa-sha256; t=1637757045; cv=pass; d=google.com; s=arc-20160816; b=m1MvDnWlbCQ+vaMoRv7JvUv61wcdAU5rPIdqugetn3tV0axrSTemX/xCayfktqE6US W210+HEr5+hXAgrKkGt3TfYnaFs1RoRpZiH0pC+LztaDS2YoW009L9WMdilIc2pLXLiU wobkKTIeu3TVuQ80tIP2/Wj/v24GPuyGVFIscMiabIfYXnlILpdSyMev8NPbTBJusDNt 4PkJMkFxDwpQfa69gn1qMDVF6V9IikI4S5v3T6otK3SWfE0W5XC6VCINwZKELsTfHwWA StnQT/GAKUTKa4M01WnoDWSvduiUaL/vpAcl+9nTKMd2LdvuOr+Oz8kzL3+zymwPIplU 65lQ== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-unsubscribe:list-subscribe:list-archive:list-help:list-post :list-id:mailing-list:precedence:content-transfer-encoding :mime-version:message-id:date:subject:cc:to:from:sender :dkim-signature; bh=8JSA0tXKbsYrwjTRppYDe4bTBAzyilwnqtIyg/Q8LgY=; b=NLuCfL5k5JU3Av8F/pHW5cEMPpx5Qw+m4zUjOShlQT+8PK5+hU1PdbZgpaq1qOx/B8 QWX7OxTM0SQ+K4num2qCEQmOVkwoU8k7lZry9OplxBOeOxvvbuXFTdXvvKe3BAVhG9PA WAbpRBaDa6+uJXqVz9CYuF21GmeaFE+oWeo5jDqILjKGr7qvXxeSjfJK2/Mr+VPMh7jC ahoGToXNa0bBWmY5PgIzT89NhWDSrevgE8xJfhqaTdBkJNgS1x8SkdCHp3DV/zgWeUSq 4Co3Un5h6NMakhW3dPhMPhGYw4B7MW8ZEaAhNlY0hsi/lI/pZqNINGtdMATMlyCz72l0 JCGw== ARC-Authentication-Results: i=2; gmr-mx.google.com; spf=neutral (google.com: 212.18.0.9 is neither permitted nor denied by domain of sbabic@denx.de) smtp.mailfrom=sbabic@denx.de DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=googlegroups.com; s=20210112; h=sender:from:to:cc:subject:date:message-id:mime-version :content-transfer-encoding:x-original-sender :x-original-authentication-results:precedence:mailing-list:list-id :list-post:list-help:list-archive:list-subscribe:list-unsubscribe; bh=8JSA0tXKbsYrwjTRppYDe4bTBAzyilwnqtIyg/Q8LgY=; b=BaKJalnxcaEETjgzUDhyvk6GEyPZRySuVa++z0jdb+P9n6lA0vWGMmBnmSWo9EcG89 iSYzJr4vqLhh5jGWm6h6CVJWWDU0CIUYaFxzWnhaXeqMa/u05NWmGNo+AelzW9pQ4wkB R+Ks1Z8s9iI56m1Yoldxzhwxi/8MLAM9bzgFQVLE3mMCnt030rkkk2TpG/Kan1uWEmZd 39PTNdoSviqBcnj540QY9Wvt4uXqLxCjJ3otUW3pSIZHQ7CMXtlIUg+N0XF38NKIhoZl ffDIpw4NaqeFiXhLqPHYCEx0ot9LKY3crdmQZ1aW/yJ0wzmf8cLKmTqNvdd374C7H1zN kl1A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=sender:x-gm-message-state:from:to:cc:subject:date:message-id :mime-version:content-transfer-encoding:x-original-sender :x-original-authentication-results:precedence:mailing-list:list-id :x-spam-checked-in-group:list-post:list-help:list-archive :list-subscribe:list-unsubscribe; bh=8JSA0tXKbsYrwjTRppYDe4bTBAzyilwnqtIyg/Q8LgY=; b=VrX9bC+n6zzwXpyJQXrxbkcFeIkIJPkek5RmNJcL+xzYyv7TgdWU+UD9ok7Uyiuk67 Aslwbd4Ty+K4ACuoQGiTV/ODwaV5pBzidA3hxhZPHvHFP3gGHw/N72Qn6jjADFmYsLw4 nG8b9cK6rPcBgjzg7slk7G4ywWbgY4UjWrSdJKXPIeLn/Iu1D7/ZtpquO7JLQL/7lPXd lNSQT2Up0QO6XqFagpObBZXySbDAM4s1X075DYZYaRGEWvPp8tSbyPWteemCAD+XLwBU fg26zSUbik9mG23e5+SNU/m8mD3WrnuqapFxkYsLZx8z0/2CqP4eZeiMUh0+/DPeNNGW OphQ== Sender: swupdate@googlegroups.com X-Gm-Message-State: AOAM532WOzO1vHNziM+5aCvax/NQumuaMG7zVeAGUn/HqcfqyW7yljmK j97/weGQI3LVZ4e319jAXf4= X-Google-Smtp-Source: ABdhPJzx9BvdNBfG0zKbVPox0Ru4L8UVT+oEwc76mvOgI31pPXn780IHGCr/X5nUGWWqSs0FrL7eWw== X-Received: by 2002:a2e:a78c:: with SMTP id c12mr14600932ljf.418.1637757044808; Wed, 24 Nov 2021 04:30:44 -0800 (PST) X-BeenThere: swupdate@googlegroups.com Received: by 2002:a05:651c:1251:: with SMTP id h17ls2264184ljh.6.gmail; Wed, 24 Nov 2021 04:30:43 -0800 (PST) X-Received: by 2002:a05:651c:154a:: with SMTP id y10mr14868336ljp.314.1637757043868; Wed, 24 Nov 2021 04:30:43 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1637757043; cv=none; d=google.com; s=arc-20160816; b=tnfUUK5Lf1bvM5NgdKgTHxc7Tyu+goh3+wUgrrXZgoVGnRxt/MCPEz2/BJwy0S76NR NiDQdTpHNZM3Ad9aQUxsZuQqebQ3X72n7SrypQ4bt2trmi0/uCy7DjeEGhpeSFCjftIh q9u4VAJdq3Fc5X1afJ/O+yzGYj3+w0SIkPnWYd6n3J+9l451BbVF5yEwylT2ZWENmoK/ cgSLZY4atYExIRzvkx+D+dV1SQAtihV17K1muh1uymTFqT2uU/k+k/XPx08uKOkM2W5k EeditWyNG1+Qh7Y0KXv8J2ggKm6OvAmSYZWcbN8P49T3aCks/dN6aKvu+7W3VJhgWTWi F26A== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:message-id:date:subject:cc :to:from; bh=cNr/COE2kqkgEU6u/UFOojMVGSkVDsyhhOI3OwBto6U=; b=dKFPVQLjnYYpq8fo9U5o4ArRMlRsGHLd1BczYfpnWOwtU/IRjC8zcQyXVJshqMhiPz PHCciYpCjS/Hfdpx3VDU2D2L08zxPmK2kZUbCBOWqjmyZlQVF/UxKPHo/UdwaKnQuZHb 0ylYsU+i3V8ghS+FZj9bMCaZae32+mRu7RNJy7DUcHI3mu4esgkDXa+FEQWVUMCkk55N iw7y4wlOx/JHuv4OjqabIo6qaTZAopEvnVXPw8Tn8AarR+Pxr4Qo+ZVb1nDdKoUMxiGJ jmA9IRBzoDJz0m9cRT3XAMeyW3dJJASmMowRBkYTKCfxMLSeu+HRkyGvrxIAvBXayJ6R qdCg== ARC-Authentication-Results: i=1; gmr-mx.google.com; spf=neutral (google.com: 212.18.0.9 is neither permitted nor denied by domain of sbabic@denx.de) smtp.mailfrom=sbabic@denx.de Received: from mail-out.m-online.net (mail-out.m-online.net. [212.18.0.9]) by gmr-mx.google.com with ESMTPS id c11si1013824ljr.8.2021.11.24.04.30.43 for (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Wed, 24 Nov 2021 04:30:43 -0800 (PST) Received-SPF: neutral (google.com: 212.18.0.9 is neither permitted nor denied by domain of sbabic@denx.de) client-ip=212.18.0.9; Received: from frontend01.mail.m-online.net (unknown [192.168.8.182]) by mail-out.m-online.net (Postfix) with ESMTP id 4HzgMl09nQz1r86h; Wed, 24 Nov 2021 13:30:42 +0100 (CET) Received: from localhost (dynscan1.mnet-online.de [192.168.6.70]) by mail.m-online.net (Postfix) with ESMTP id 4HzgMk5NmZz1qqkB; Wed, 24 Nov 2021 13:30:42 +0100 (CET) X-Virus-Scanned: amavisd-new at mnet-online.de Received: from mail.mnet-online.de ([192.168.8.182]) by localhost (dynscan1.mail.m-online.net [192.168.6.70]) (amavisd-new, port 10024) with ESMTP id 49tg5S10dTKB; Wed, 24 Nov 2021 13:30:41 +0100 (CET) Received: from babic.homelinux.org (host-88-217-136-221.customer.m-online.net [88.217.136.221]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.mnet-online.de (Postfix) with ESMTPS; Wed, 24 Nov 2021 13:30:40 +0100 (CET) Received: from localhost (mail.babic.homelinux.org [127.0.0.1]) by babic.homelinux.org (Postfix) with ESMTP id 1C3FB45412CC; Wed, 24 Nov 2021 13:30:40 +0100 (CET) X-Virus-Scanned: Debian amavisd-new at babic.homelinux.org Received: from babic.homelinux.org ([127.0.0.1]) by localhost (mail.babic.homelinux.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 705PDecZiNfK; Wed, 24 Nov 2021 13:30:37 +0100 (CET) Received: from paperino.fritz.box (paperino.fritz.box [192.168.178.48]) by babic.homelinux.org (Postfix) with ESMTP id 0CEFC4541280; Wed, 24 Nov 2021 13:30:37 +0100 (CET) From: Stefano Babic To: swupdate@googlegroups.com Cc: Stefano Babic , Dominique Martinet Subject: [swupdate] [PATCH] doc: fix and extend delta update Date: Wed, 24 Nov 2021 13:30:26 +0100 Message-Id: <20211124123026.125205-1-sbabic@denx.de> X-Mailer: git-send-email 2.25.1 MIME-Version: 1.0 X-Original-Sender: sbabic@denx.de X-Original-Authentication-Results: gmr-mx.google.com; spf=neutral (google.com: 212.18.0.9 is neither permitted nor denied by domain of sbabic@denx.de) smtp.mailfrom=sbabic@denx.de Precedence: list Mailing-list: list swupdate@googlegroups.com; contact swupdate+owners@googlegroups.com List-ID: X-Spam-Checked-In-Group: swupdate@googlegroups.com X-Google-Group-Id: 605343134186 List-Post: , List-Help: , List-Archive: , List-Unsubscribe: , Remove note for Zchunk branch because it was merged into mainline. Add some hints how to build the ZCK files. Signed-off-by: Stefano Babic Reported-by: Dominique Martinet --- doc/source/delta-update.rst | 48 +++++++++++++++++++++++++++++++++---- 1 file changed, 44 insertions(+), 4 deletions(-) diff --git a/doc/source/delta-update.rst b/doc/source/delta-update.rst index 5307dcd..9327315 100644 --- a/doc/source/delta-update.rst +++ b/doc/source/delta-update.rst @@ -162,10 +162,8 @@ A short list of changes in the Zchunk project is: - add API to allow an external caller to take itself the decision if a chunk must be downloaded or reused. -Some of these changes were merged into Zchunk project, some are still open. Zchunk working with -SWUpdate ist stored on a separate branch_. - -.. _branch: https://github.com/sbabic/zchunk/tree/devel +These changes were merged into Zchunk project - be sure to get a recent version of Zchunk, at least with +commit 1b36f8b5e0ecb, that means newer as 1.1.16. Most of missing features in Zchunk listed in TODO for the project have no relevance here: SWUpdate already verifies the downloaded data, and there is no need to add signatures to Zchunk itself. @@ -221,3 +219,45 @@ several requests to the downloader, and tracking each of them. The downloader is thought as dummy servant: it starts the connection, retrieves HTTP headers and data, and sends them back to the caller. The delta handler is then responsible to parse the answer, and to retrieve the missing chunks from the multipart HTTP body. + +Creation of ZCK Header and ZCK file for SWUpdate +------------------------------------------------ + +Zchunk supports more SHA algorithm and it sets as default SHA512/128. This is not compatible with SWUpdate +that just support SHA256. Be sure to generate header and chunks with SHA256 support. +You have to enable the generation of hashes for uncompressed chunk, too. A possible usage of the tool +is: + +:: + + zck --output -u --chunk-hash-type sha256 + +The output is the ZCK file with all chunks. +This file should be put on a Webserver accessible to the target, and that supports Range Request +(RFC 7233). All modern Webserver support it. + +The SWU must just contain the header. This can be easy extracted from the ZCK file with: + +:: + + HSIZE=`zck_read_header -v | grep "Header size" | cut -d':' -f2` + dd if= of= bs=1 count=$((HSIZE)) + +Using ZCK tools to foresee download size +---------------------------------------- + +There are tools that can be used at build time to know how many chunks should be downloaded +when a device is upgrading from a known version. You can use `zck_cmp_uncomp` from the test +directory: + +:: + + ../build/test/zck_cmp_uncomp --verbose + +This prints a list with all chunks, marking them with SRC if they are the same in the old version +and they should not retrieved and with DST if they are new and must be downloaded. +The tool show at the end a summary with the total number of bytes of the new release (uncompressed) and how +many bytes must be downloaded for the upgrade. +Please remmeber that these value are just payload. SWUpdate reports a summary, too, but it takes into account +alöso the HTTP overhead (headers, etc.), so that values are not the same and the ones from SWUpdate are +slightly bigger.