From patchwork Mon Oct 11 11:21:56 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Stefano Babic X-Patchwork-Id: 1539258 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: bilbo.ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=googlegroups.com header.i=@googlegroups.com header.a=rsa-sha256 header.s=20210112 header.b=laaaWaUi; dkim-atps=neutral Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=googlegroups.com (client-ip=2a00:1450:4864:20::140; helo=mail-lf1-x140.google.com; envelope-from=swupdate+bncbcxploxj6ikrbcn5scfqmgqekwzcqra@googlegroups.com; receiver=) Received: from mail-lf1-x140.google.com (mail-lf1-x140.google.com [IPv6:2a00:1450:4864:20::140]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by bilbo.ozlabs.org (Postfix) with ESMTPS id 4HSbxm6jRHz9sPB for ; Mon, 11 Oct 2021 22:22:52 +1100 (AEDT) Received: by mail-lf1-x140.google.com with SMTP id o5-20020ac24c45000000b003fd932e4642sf1330770lfk.14 for ; Mon, 11 Oct 2021 04:22:52 -0700 (PDT) ARC-Seal: i=2; a=rsa-sha256; t=1633951370; cv=pass; d=google.com; s=arc-20160816; b=WGEYtE1ZBdGGsmEClpdBfDUxCCI7tR8bpfH7tdTFaMT3DH3p2BisanX1TcXcOgq130 NMqmeNFzmon7QjPkHBOBfFXNh+GQb2CZ5RBTrMUZtUV8TvW4HNBoJGJjT44pYO/2KJba 5+TbsrS7COtAS9BzPRKIydIgOk3Wh2BA8KIfJNus++/NVBtL3gkIHUETBK45RSRBaKkI IqAKKujCWHCr2mHqooBqaS9P1T7MrPxE8hYQjBuoJq/Bd22dcvTEIefbaORSsP5mX22x HzgJpyxymjT0K9uH8G8AdUa850cNHOrjGoyoL7n6UdJTyuXFImr8HaiqNb4mm/AQaWYO e0uQ== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-unsubscribe:list-subscribe:list-archive:list-help:list-post :list-id:mailing-list:precedence:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:sender:dkim-signature; bh=QlpQyVJ9cfDxJ2XNhgu3vR8hLL/6hT8PUPXZQ1+ld/w=; b=ep2BnhoUoFvSpS1/F4if6B1X+gjHx65ZOYw83W40NV6yy1OlT2BAiOLOwD15Xm1c6g Vpo6whtNraORT5Re8tC2HBHEt8cZ4owRDr2WSIPmZeuljnkBMbRP6+KDy4OPsAnAk+4G jA9l61cpSeaMF9Ql/3c6acQlAXAzAc2BLmJG/IQKaJfPIpz9DDlf2Q4i5O7CmzVO2kM/ xNMnr3Ic/+NPGqAusNawLy2n8+Q2rEyw8rkvPup6ag3OlivgmhPUnau/0UEU2wDMfKDZ Wew/Zy1ceczB8bq9eZAQAujP82o9ULGGuVOmjpQEVcOCnvjIMxcpao6zrHPyTRL4VXJU Gw6g== ARC-Authentication-Results: i=2; gmr-mx.google.com; spf=neutral (google.com: 212.18.0.9 is neither permitted nor denied by domain of sbabic@denx.de) smtp.mailfrom=sbabic@denx.de DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=googlegroups.com; s=20210112; h=sender:from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:x-original-sender:x-original-authentication-results :precedence:mailing-list:list-id:list-post:list-help:list-archive :list-subscribe:list-unsubscribe; bh=QlpQyVJ9cfDxJ2XNhgu3vR8hLL/6hT8PUPXZQ1+ld/w=; b=laaaWaUi8R17W9sourDIXDvjUP7BgjCU9alAq9LZi6be4Auhc/BrxlqPHq3+Gl/lXh 1rvJBTUuwySgaoHagc/fmtLra0mtiUjM0cbVDK7cCHVB4o5piEUTCgEOM4yhWIChn718 Umb8yZ+olusxmy1tb1BpNIMmZR2O40ZhRuzl53pA0XvypYH3P9SlBF3XyyeK1HBP3p8Y H1GTFjVy/Ichv0fLnAzKQoY5jCJsG+t4W60jxVr5QGd8p3+glMti/rBXLF2wzQeUs0p9 ekqOJTKXjzq/3oGwwbxszX9EvIeLk5k52QQ5+PIbqzHSIpeqR1Ya9AS1C7RfdqA5H1Q1 MFqQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=sender:x-gm-message-state:from:to:cc:subject:date:message-id :in-reply-to:references:mime-version:x-original-sender :x-original-authentication-results:precedence:mailing-list:list-id :x-spam-checked-in-group:list-post:list-help:list-archive :list-subscribe:list-unsubscribe; bh=QlpQyVJ9cfDxJ2XNhgu3vR8hLL/6hT8PUPXZQ1+ld/w=; b=54ml9fuLnwpdJB4ZmDQc+qQaaYg8Avdhs+VBLrOqOIQyuBn81ewfGMy+4/DWy+0jog Evv0Sr0RPLQbgfyar+FSP92qpuc03iWVRZM5dzB+/59luM5RC4b5gMh6zqldkRXHq69W 1nHbrzHXQ4v/mWuwI0oX8Shrgl8N9O5JPQ3QRhiB760XXsQso+LBf4ITi5RZ7hPon5Ms gWa0rJuBJG1/2Sx27QPKoEtXVVMFv8A16T//3cWwmPhfXRxXTKvK3yuHTmcn8la6urVs dS1igIgh5dAcwQdNZcp8TkOmJaPHG1aHG2GsmEKANHudlROvRnye8o7vbc7/1PgeUwPs 2gPA== Sender: swupdate@googlegroups.com X-Gm-Message-State: AOAM533VzfRUdoae2wabCQLZvUWKGj7AhhQg43grkEKFbJSkvhYhlL8O WOXos5kSlZ5jR+ieKd6zOBQ= X-Google-Smtp-Source: ABdhPJxgWikGH9iTYzdRwzYqPkrRn0AV/P1ImKU5rY/jcE1K9p/lH+PwY2AEaioRA51sU5X+ehjGSQ== X-Received: by 2002:a05:6512:b0f:: with SMTP id w15mr26807204lfu.164.1633951369869; Mon, 11 Oct 2021 04:22:49 -0700 (PDT) X-BeenThere: swupdate@googlegroups.com Received: by 2002:a2e:a274:: with SMTP id k20ls1665921ljm.11.gmail; Mon, 11 Oct 2021 04:22:49 -0700 (PDT) X-Received: by 2002:a05:651c:211a:: with SMTP id a26mr22316097ljq.148.1633951369000; Mon, 11 Oct 2021 04:22:49 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1633951368; cv=none; d=google.com; s=arc-20160816; b=mfQChOwCdmp3Tf/KjSlF86IaNUlDelpMkb/YI+P4QpZMy3U2x63LT9Xca5OW+7OcCD lOuhuxDnIPYuZxzt2MIfkArHPPZmSbbiLy4+UsJZHXqNnM+szSYp49z4hNoryEUOU1z/ cjzT10YYF/Q+uYsicQUoiyP9keJOHjHatjDLYPwljr3PHwk+cVh6J+ATyie+EWu8E37r 2y+PIOee7IYwAyiViacj85RHV7mVSksVIsV/Jj5IuTJV5ijTBgwtkC8osmwk6R10rLB8 YfbVoF5VKrbfArm1LO+dIsfuGZFXkWilD4nUFd4gcwxK4Y+U4Uk95ldpxsI49yVW5rlp wlZw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from; bh=imtLbsyt+nacG1MnCMG9R160n5EmhUNJXPfsHuy8B5c=; b=V2k7qmVouuqenNaglZCyQBXI91KLq7KWfo+e4c7T7KmjU6nOcRuLPQ88IkkP4322uG 73jkbXSvLaV9FC+9QmjCc0JDg6+zboRlZOCm9XVIeOthJ9cMb4YE0193v24yFnjIKa+b EhrK2qXb5KqTWH1D0CKO/tNOwpoP8yuobK0pX+abhx3rF9B1toLRyydYcHfC/zDDuVzA jrq4cPrG1zaPsvCA1U4oJMwDtLkDyy4F5QSIhA+c/W7LGAVg1kHHixcDNutHs/ur/7q/ ovN/eBFwRICdH6N/ceFQ0aXvJtnC4NbsW4JnycIUe2H1RgFK1XzLObovkcAL5Zei5RnL YWww== ARC-Authentication-Results: i=1; gmr-mx.google.com; spf=neutral (google.com: 212.18.0.9 is neither permitted nor denied by domain of sbabic@denx.de) smtp.mailfrom=sbabic@denx.de Received: from mail-out.m-online.net (mail-out.m-online.net. [212.18.0.9]) by gmr-mx.google.com with ESMTPS id s16si613668lfp.6.2021.10.11.04.22.48 for (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Mon, 11 Oct 2021 04:22:48 -0700 (PDT) Received-SPF: neutral (google.com: 212.18.0.9 is neither permitted nor denied by domain of sbabic@denx.de) client-ip=212.18.0.9; Received: from frontend01.mail.m-online.net (unknown [192.168.8.182]) by mail-out.m-online.net (Postfix) with ESMTP id 4HSbxh3nPqz1qwGj; Mon, 11 Oct 2021 13:22:48 +0200 (CEST) Received: from localhost (dynscan1.mnet-online.de [192.168.6.70]) by mail.m-online.net (Postfix) with ESMTP id 4HSbxh3h9jz1qqkB; Mon, 11 Oct 2021 13:22:48 +0200 (CEST) X-Virus-Scanned: amavisd-new at mnet-online.de Received: from mail.mnet-online.de ([192.168.8.182]) by localhost (dynscan1.mail.m-online.net [192.168.6.70]) (amavisd-new, port 10024) with ESMTP id JN_OOPbpVdjz; Mon, 11 Oct 2021 13:22:47 +0200 (CEST) Received: from babic.homelinux.org (host-88-217-136-221.customer.m-online.net [88.217.136.221]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.mnet-online.de (Postfix) with ESMTPS; Mon, 11 Oct 2021 13:22:47 +0200 (CEST) Received: from localhost (mail.babic.homelinux.org [127.0.0.1]) by babic.homelinux.org (Postfix) with ESMTP id 59D464540431; Mon, 11 Oct 2021 13:22:47 +0200 (CEST) X-Virus-Scanned: Debian amavisd-new at babic.homelinux.org Received: from babic.homelinux.org ([127.0.0.1]) by localhost (mail.babic.homelinux.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id GzDfSILmyAlw; Mon, 11 Oct 2021 13:22:44 +0200 (CEST) Received: from paperino.fritz.box (paperino.fritz.box [192.168.178.48]) by babic.homelinux.org (Postfix) with ESMTP id 6AAEC4542813; Mon, 11 Oct 2021 13:22:07 +0200 (CEST) From: Stefano Babic To: swupdate@googlegroups.com Cc: Stefano Babic Subject: [swupdate] [PATCH 29/29] delta: add the option to limit size for source Date: Mon, 11 Oct 2021 13:21:56 +0200 Message-Id: <20211011112156.44192-30-sbabic@denx.de> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20211011112156.44192-1-sbabic@denx.de> References: <20211011112156.44192-1-sbabic@denx.de> MIME-Version: 1.0 X-Original-Sender: sbabic@denx.de X-Original-Authentication-Results: gmr-mx.google.com; spf=neutral (google.com: 212.18.0.9 is neither permitted nor denied by domain of sbabic@denx.de) smtp.mailfrom=sbabic@denx.de Precedence: list Mailing-list: list swupdate@googlegroups.com; contact swupdate+owners@googlegroups.com List-ID: X-Spam-Checked-In-Group: swupdate@googlegroups.com X-Google-Group-Id: 605343134186 List-Post: , List-Help: , List-Archive: , List-Unsubscribe: , When upgrading a partition, the filesystem could be much more smaller as the partition itself. The handler as default will read the whole partition and it will create the whole ZCK index for it, but this requires more memory as really needed. Add a way to check the real size of the filesystem, and do not index the rest of the partition. Signed-off-by: Stefano Babic --- doc/source/handlers.rst | 6 ++++++ handlers/delta_handler.c | 39 +++++++++++++++++++++++++++++++++++++-- 2 files changed, 43 insertions(+), 2 deletions(-) diff --git a/doc/source/handlers.rst b/doc/source/handlers.rst index 54a9b6c..d5ddd6b 100644 --- a/doc/source/handlers.rst +++ b/doc/source/handlers.rst @@ -972,6 +972,12 @@ The resulting header file must be packed inside the SWU. | | | printed, and it reports if a chunk | | | | is downloaded or copied from the source. | +-------------+-------------+----------------------------------------------------+ + | source-sitze| string | This limits the index of the source | + | | | It is helpful in case of filesystem in much | + | | | bigger partition. It has the value for the size | + | | | or it can be set to "detect" and the handler | + | | | will try to find the effective size of fs. | + +-------------+-------------+----------------------------------------------------+ Example: diff --git a/handlers/delta_handler.c b/handlers/delta_handler.c index e0202ab..0a59a59 100644 --- a/handlers/delta_handler.c +++ b/handlers/delta_handler.c @@ -21,6 +21,7 @@ #include #include #include +#include #include #include #include @@ -35,6 +36,7 @@ #include #include #include +#include #include "delta_handler.h" #include "multipart_parser.h" #include "installer.h" @@ -89,6 +91,8 @@ struct hnd_priv { char *srcdev; /* device as source for comparison */ char *chainhandler; /* Handler to pass the decompressed image */ zck_log_type zckloglevel; /* if found, set log level for ZCK to this */ + bool detectsrcsize; /* if set, try to compute size of filesystem in srcdev */ + size_t srcsize; /* Size of source */ unsigned long max_ranges; /* Max allowed ranges (configured via sw-description) */ /* Data to be transferred to chain handler */ struct img_type img; @@ -319,6 +323,15 @@ static int delta_retrieve_attributes(struct img_type *img, struct hnd_priv *priv if (errno || priv->max_ranges == 0) priv->max_ranges = DEFAULT_MAX_RANGES; + char *srcsize; + srcsize = dict_get_value(&img->properties, "source-size"); + if (srcsize) { + if (!strcmp(srcsize, "detect")) + priv->detectsrcsize = true; + else + priv->srcsize = ustrtoull(srcsize, 10); + } + char *zckloglevel = dict_get_value(&img->properties, "zckloglevel"); if (!zckloglevel) return 0; @@ -452,7 +465,7 @@ static void zck_log_toswupdate(const char *function, zck_log_type lt, /* * Create a zck Index from a file */ -static bool create_zckindex(zckCtx *zck, int fd) +static bool create_zckindex(zckCtx *zck, int fd, size_t maxbytes) { const size_t bufsize = 16384; char *buf = malloc(bufsize); @@ -470,6 +483,8 @@ static bool create_zckindex(zckCtx *zck, int fd) free(buf); return false; } + if (maxbytes && n > maxbytes) + break; } free(buf); @@ -917,6 +932,26 @@ static int install_delta(struct img_type *img, ERROR("/dev/null not present or cannot be opened, aborting..."); goto cleanup; } + + char *filesystem; + if (priv->detectsrcsize) { + filesystem = diskformat_fs_detect(priv->srcdev); + if (filesystem) { + char* DATADST_DIR = alloca(strlen(get_tmpdir())+strlen(DATADST_DIR_SUFFIX)+1); + sprintf(DATADST_DIR, "%s%s", get_tmpdir(), DATADST_DIR_SUFFIX); + if (!swupdate_mount(priv->srcdev, DATADST_DIR, filesystem)) { + struct statvfs vfs; + if (!statvfs(DATADST_DIR, &vfs)) { + TRACE("Detected filesystem %s, block size : %lu, %lu blocks = %lu size", + filesystem, vfs.f_frsize, vfs.f_blocks, vfs.f_frsize * vfs.f_blocks); + priv->srcsize = vfs.f_frsize * vfs.f_blocks; + } + swupdate_umount(DATADST_DIR); + } + free(filesystem); + } + } + in_fd = open(priv->srcdev, O_RDONLY); if(in_fd < 0) { ERROR("Unable to open Source : %s for reading", priv->srcdev); @@ -983,7 +1018,7 @@ static int install_delta(struct img_type *img, goto cleanup; } - if (!create_zckindex(zckSrc, in_fd)) { + if (!create_zckindex(zckSrc, in_fd, priv->srcsize)) { WARN("ZCK Header form %s cannot be created, fallback to full download", priv->srcdev); } else {