From patchwork Sun Nov 14 17:27:31 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Stefano Babic X-Patchwork-Id: 1554927 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: bilbo.ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=googlegroups.com header.i=@googlegroups.com header.a=rsa-sha256 header.s=20210112 header.b=jG+lmmm1; dkim-atps=neutral Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=googlegroups.com (client-ip=2a00:1450:4864:20::53f; helo=mail-ed1-x53f.google.com; envelope-from=swupdate+bncbcxploxj6ikrb74rywgamgqe4kem6sq@googlegroups.com; receiver=) Received: from mail-ed1-x53f.google.com (mail-ed1-x53f.google.com [IPv6:2a00:1450:4864:20::53f]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by bilbo.ozlabs.org (Postfix) with ESMTPS id 4Hsfch2FT1z9sPf for ; Mon, 15 Nov 2021 04:36:03 +1100 (AEDT) Received: by mail-ed1-x53f.google.com with SMTP id l15-20020a056402124f00b003e57269ab87sf5439129edw.6 for ; Sun, 14 Nov 2021 09:36:03 -0800 (PST) ARC-Seal: i=2; a=rsa-sha256; t=1636911359; cv=pass; d=google.com; s=arc-20160816; b=tTzklSPRvPUwZ0p265cLvSj69bXLnNL+uUmhPYENLw4CYejYrViWMFDwsQXrl4C+ye ZyE23CyvgV1I3BESPITzmtvx89gK9JdWu6UnnUZaqmsBSE8NvHn0S3aV+PhZ8uKBVVB1 kwdmm0DGIL6/ry4QuFXJXQ/Y5pUFYV9R9z+m5loAucnXnRLbv8CbpQ3DiUgxLB7p2aHS PObPSDMQUmdzFSePtW/YoQxwQ7ANT3higtMlWbq2K26T/VcJE0tL42yb1GIxyI8TcYxK R2edu0KTUvrzrA3iCWcXcAYdgIphUHL+TWcD+gXDQ/3E/JVD+y51N7A9lTPPapzjLs01 e0tQ== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-unsubscribe:list-subscribe:list-archive:list-help:list-post :list-id:mailing-list:precedence:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:sender:dkim-signature; bh=6XXVraxsnvYMSDx5DIEKRznswtnE1RuweU++plG/pJc=; b=ocKslRgBNg80BKDijatNMKLMibE0xANKGXQuaaX4N9Y/TIDcsP2CDmi5u1rCD0yTWG j3I4+j3z3Hc8FHAviaguMwLPNjKVRwBTX3ZTIr0Y+3UPbhFSSJJme65+jM8ctJe4BOA8 +U/dNjkTTGV0/zBLAoeY42E4Sjv0lwvdr8Ro3ACaU+mTQtD7jZcXWRXLNMX4QvUKXlfC icxMYXwRf6c+rkWOz194pVV5dgLC1/PxXkilJ2GHWCQgohFyeqFDYGTnk3+m3KmnmHSM 0VE4h3iw1XfUgqVl4lnpv/UpQqObFyrjdyAWt4E7AjJSfwap8NY10yEx7gEOikm53rcV epEw== ARC-Authentication-Results: i=2; gmr-mx.google.com; spf=neutral (google.com: 212.18.0.9 is neither permitted nor denied by domain of sbabic@denx.de) smtp.mailfrom=sbabic@denx.de DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=googlegroups.com; s=20210112; h=sender:from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:x-original-sender:x-original-authentication-results :precedence:mailing-list:list-id:list-post:list-help:list-archive :list-subscribe:list-unsubscribe; bh=6XXVraxsnvYMSDx5DIEKRznswtnE1RuweU++plG/pJc=; b=jG+lmmm1RnQeonA5PHhDj/Tj1ZgRFopQjM+mluS5+gPd6YvSU0HIhAuOFF4f549R0f 13aW0xCygSCnCIoX5TE3qFAHPOYBXxsVcDrVrobJrAojVzOlDwUyU8XhZGrorGBfV255 MKSitak+v9R6vyxpFlKsHuY497O/DidNziZDijx5HgW5MO4A3C8dQFOCBIVyaP2UOTVI l76448C3W16TJsHaOhTKX1JEVGvIMGtvZnI/ay8MNDn1chhVPS+X1syRZodLvn6jkjH4 nmCVxcieTL4WzB3aZEMGquq7vUSqM2eCsPB/BsWy3093cKPbRwu3IAiQNzVe1JiciMYd pY1A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=sender:x-gm-message-state:from:to:cc:subject:date:message-id :in-reply-to:references:mime-version:x-original-sender :x-original-authentication-results:precedence:mailing-list:list-id :x-spam-checked-in-group:list-post:list-help:list-archive :list-subscribe:list-unsubscribe; bh=6XXVraxsnvYMSDx5DIEKRznswtnE1RuweU++plG/pJc=; b=T330Woandj7nhHtBZXGWA8+qbmjwCAXStl+giEbIMsUXFRcQLXFJ77tl2qBCV7XTAX cigKVcnyu4bfOCUZY7Dk2MYf9EYBPboYQ+zlPLKOJjOZ3gQV6a2kLBNUp/OPeLinsSzM OZ0bzJPsbqPIegrr2v2AAfPqds4PkOgWRx0xe/2r4oiaJnlMmqc2pH8sqq9Xo3g3oieS KVeAAWWLeu7pGiYOyRxQBOC3pxscCIj44GXt3AbwRbGRNI7SAnczHTLr2eO+tEV0ttdW NVF2+7jOdLDg7qq7vGQTufY3cOrHmBc080dn1Jr5+UJC//FrDmqh8oWLoVFHBbtcr4Ri NO2A== Sender: swupdate@googlegroups.com X-Gm-Message-State: AOAM532N/wIbXA1U4LAtYVkC+rPAQZPzc787Wk48KTW93KCSMjYaL9de hiO026uJWw+BySDhX/SMiQA= X-Google-Smtp-Source: ABdhPJwzRs6cpmHa7ml3PwT8mh8w4u/jo+jVWn/JWk/zDPCEfutV3kSAaKNZ2cS5tG3weUq9y8ooug== X-Received: by 2002:a17:907:3e25:: with SMTP id hp37mr41579958ejc.43.1636911359435; Sun, 14 Nov 2021 09:35:59 -0800 (PST) X-BeenThere: swupdate@googlegroups.com Received: by 2002:a50:8e41:: with SMTP id 1ls3830438edx.0.gmail; Sun, 14 Nov 2021 09:35:58 -0800 (PST) X-Received: by 2002:a05:6402:3586:: with SMTP id y6mr45944651edc.332.1636911358747; Sun, 14 Nov 2021 09:35:58 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1636911358; cv=none; d=google.com; s=arc-20160816; b=UrhYa1HZrxZDu/HzYbgkpl3MYwbs3RpHjGsBSmUgJbJWiShxriHJ6/8Gk4XJVj94qC 9OtkjzC1iEw/z1wTI7cu5j76J85Vw4SM37EQesdxjdWbiEIwuSdfWFSWA8pDmHeyqqFU ibOG+yiNgASoOe4pEn/RkGPPSuumIQWt+L7TRQe2WFyDxbV421kRHklphboH6zvCZ07o WGtPCnvniu5F1FnoepAnLoGOA5tI4CgEeD5GoZQEohDEvNEA9T4IZQCfqCvkhg9ZGplu 2FMJPSG37b8iQjeZWjLOhybPCAD/4f8HTIHuwXv/WLWTzid4ncp0Ll4H0DiY1+//MNHB 33kw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from; bh=f23ApHvZEI02DrFW3Hs2o9GE+X3G05zm3O6UO6Q0qRk=; b=mVrXRS94r0p+7ejGsSF+ZaG3StfqV8+luK5GSM0ZvPr+PtQOfNOGfA0RHyrqjNaWvs nfOwNXtxlJUFPTMGMT7/9Uihmzj3MsfPQJ2mUt+BnuLy1fkWpEEizaMPdpzjGjwJUw37 07V9aWsGuTjwU16B+8x4r5hlNXFHRvruMeLCHz694gwZVVM3jcXT3t7vJbmT4YyePo72 A6fcr+AydcBoBbVpBqzz0oKiwAw4y0ehdZMt4fuo7CUn80RrxrOLRBLMXS5U6PK+QnCL PLpocTJzRCvLcyfTIEp14vzK20ci/X/Sx6VHyEwDXl+ZI0J35FwMOKkKJfrToTm1SF67 kKqA== ARC-Authentication-Results: i=1; gmr-mx.google.com; spf=neutral (google.com: 212.18.0.9 is neither permitted nor denied by domain of sbabic@denx.de) smtp.mailfrom=sbabic@denx.de Received: from mail-out.m-online.net (mail-out.m-online.net. [212.18.0.9]) by gmr-mx.google.com with ESMTPS id u19si1155525edo.5.2021.11.14.09.35.58 for (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Sun, 14 Nov 2021 09:35:58 -0800 (PST) Received-SPF: neutral (google.com: 212.18.0.9 is neither permitted nor denied by domain of sbabic@denx.de) client-ip=212.18.0.9; Received: from frontend01.mail.m-online.net (unknown [192.168.8.182]) by mail-out.m-online.net (Postfix) with ESMTP id 4HsfcZ3ZDHz1qwdZ; Sun, 14 Nov 2021 18:35:58 +0100 (CET) Received: from localhost (dynscan1.mnet-online.de [192.168.6.70]) by mail.m-online.net (Postfix) with ESMTP id 4HsfcZ3SVhz1qqkB; Sun, 14 Nov 2021 18:35:58 +0100 (CET) X-Virus-Scanned: amavisd-new at mnet-online.de Received: from mail.mnet-online.de ([192.168.8.182]) by localhost (dynscan1.mail.m-online.net [192.168.6.70]) (amavisd-new, port 10024) with ESMTP id cwhXLIWcHDP9; Sun, 14 Nov 2021 18:35:57 +0100 (CET) Received: from babic.homelinux.org (host-88-217-136-221.customer.m-online.net [88.217.136.221]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.mnet-online.de (Postfix) with ESMTPS; Sun, 14 Nov 2021 18:35:57 +0100 (CET) Received: from localhost (mail.babic.homelinux.org [127.0.0.1]) by babic.homelinux.org (Postfix) with ESMTP id 8361B4541BC4; Sun, 14 Nov 2021 18:28:47 +0100 (CET) X-Virus-Scanned: Debian amavisd-new at babic.homelinux.org Received: from babic.homelinux.org ([127.0.0.1]) by localhost (mail.babic.homelinux.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id tgU0PnJdUppB; Sun, 14 Nov 2021 18:28:44 +0100 (CET) Received: from paperino.fritz.box (paperino.fritz.box [192.168.178.48]) by babic.homelinux.org (Postfix) with ESMTP id 5588E4542939; Sun, 14 Nov 2021 18:27:47 +0100 (CET) From: Stefano Babic To: swupdate@googlegroups.com Cc: Stefano Babic Subject: [swupdate] [PATCH V2 34/36] delta: add the option to limit size for source Date: Sun, 14 Nov 2021 18:27:31 +0100 Message-Id: <20211114172733.71602-35-sbabic@denx.de> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20211114172733.71602-1-sbabic@denx.de> References: <20211114172733.71602-1-sbabic@denx.de> MIME-Version: 1.0 X-Original-Sender: sbabic@denx.de X-Original-Authentication-Results: gmr-mx.google.com; spf=neutral (google.com: 212.18.0.9 is neither permitted nor denied by domain of sbabic@denx.de) smtp.mailfrom=sbabic@denx.de Precedence: list Mailing-list: list swupdate@googlegroups.com; contact swupdate+owners@googlegroups.com List-ID: X-Spam-Checked-In-Group: swupdate@googlegroups.com X-Google-Group-Id: 605343134186 List-Post: , List-Help: , List-Archive: , List-Unsubscribe: , When upgrading a partition, the filesystem could be much more smaller as the partition itself. The handler as default will read the whole partition and it will create the whole ZCK index for it, but this requires more memory as really needed. Add a way to check the real size of the filesystem, and do not index the rest of the partition. Signed-off-by: Stefano Babic --- doc/source/handlers.rst | 6 ++++++ handlers/delta_handler.c | 42 ++++++++++++++++++++++++++++++++++++++-- 2 files changed, 46 insertions(+), 2 deletions(-) diff --git a/doc/source/handlers.rst b/doc/source/handlers.rst index 8d78bbd..ecd2bf5 100644 --- a/doc/source/handlers.rst +++ b/doc/source/handlers.rst @@ -996,6 +996,12 @@ The resulting header file must be packed inside the SWU. | | | printed, and it reports if a chunk | | | | is downloaded or copied from the source. | +-------------+-------------+----------------------------------------------------+ + | source-size | string | This limits the index of the source | + | | | It is helpful in case of filesystem in much | + | | | bigger partition. It has the value for the size | + | | | or it can be set to "detect" and the handler | + | | | will try to find the effective size of fs. | + +-------------+-------------+----------------------------------------------------+ Example: diff --git a/handlers/delta_handler.c b/handlers/delta_handler.c index d959509..5229657 100644 --- a/handlers/delta_handler.c +++ b/handlers/delta_handler.c @@ -21,6 +21,7 @@ #include #include #include +#include #include #include #include @@ -35,6 +36,7 @@ #include #include #include +#include #include "delta_handler.h" #include "multipart_parser.h" #include "installer.h" @@ -90,6 +92,8 @@ struct hnd_priv { char *srcdev; /* device as source for comparison */ char *chainhandler; /* Handler to pass the decompressed image */ zck_log_type zckloglevel; /* if found, set log level for ZCK to this */ + bool detectsrcsize; /* if set, try to compute size of filesystem in srcdev */ + size_t srcsize; /* Size of source */ unsigned long max_ranges; /* Max allowed ranges (configured via sw-description) */ /* Data to be transferred to chain handler */ struct img_type img; @@ -319,6 +323,15 @@ static int delta_retrieve_attributes(struct img_type *img, struct hnd_priv *priv if (errno || priv->max_ranges == 0) priv->max_ranges = DEFAULT_MAX_RANGES; + char *srcsize; + srcsize = dict_get_value(&img->properties, "source-size"); + if (srcsize) { + if (!strcmp(srcsize, "detect")) + priv->detectsrcsize = true; + else + priv->srcsize = ustrtoull(srcsize, 10); + } + char *zckloglevel = dict_get_value(&img->properties, "zckloglevel"); if (!zckloglevel) return 0; @@ -452,7 +465,7 @@ static void zck_log_toswupdate(const char *function, zck_log_type lt, /* * Create a zck Index from a file */ -static bool create_zckindex(zckCtx *zck, int fd) +static bool create_zckindex(zckCtx *zck, int fd, size_t maxbytes) { const size_t bufsize = 16384; char *buf = malloc(bufsize); @@ -470,6 +483,8 @@ static bool create_zckindex(zckCtx *zck, int fd) free(buf); return false; } + if (maxbytes && n > maxbytes) + break; } free(buf); @@ -918,6 +933,29 @@ static int install_delta(struct img_type *img, ERROR("/dev/null not present or cannot be opened, aborting..."); goto cleanup; } + + if (priv->detectsrcsize) { +#if defined(CONFIG_DISKFORMAT) + char *filesystem = diskformat_fs_detect(priv->srcdev); + if (filesystem) { + char* DATADST_DIR = alloca(strlen(get_tmpdir())+strlen(DATADST_DIR_SUFFIX)+1); + sprintf(DATADST_DIR, "%s%s", get_tmpdir(), DATADST_DIR_SUFFIX); + if (!swupdate_mount(priv->srcdev, DATADST_DIR, filesystem)) { + struct statvfs vfs; + if (!statvfs(DATADST_DIR, &vfs)) { + TRACE("Detected filesystem %s, block size : %lu, %lu blocks = %lu size", + filesystem, vfs.f_frsize, vfs.f_blocks, vfs.f_frsize * vfs.f_blocks); + priv->srcsize = vfs.f_frsize * vfs.f_blocks; + } + swupdate_umount(DATADST_DIR); + } + free(filesystem); + } +#else + WARN("SWUPdate not compiled with DISKFORMAT, skipping size detection.."); +#endif + } + in_fd = open(priv->srcdev, O_RDONLY); if(in_fd < 0) { ERROR("Unable to open Source : %s for reading", priv->srcdev); @@ -984,7 +1022,7 @@ static int install_delta(struct img_type *img, goto cleanup; } - if (!create_zckindex(zckSrc, in_fd)) { + if (!create_zckindex(zckSrc, in_fd, priv->srcsize)) { WARN("ZCK Header form %s cannot be created, fallback to full download", priv->srcdev); } else {