From patchwork Mon Aug 24 23:00:28 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Baptiste Jonglez X-Patchwork-Id: 1350729 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=lists.openwrt.org (client-ip=2001:8b0:10b:1231::1; helo=merlin.infradead.org; envelope-from=openwrt-devel-bounces+incoming=patchwork.ozlabs.org@lists.openwrt.org; receiver=) Authentication-Results: ozlabs.org; dmarc=none (p=none dis=none) header.from=bitsofnetworks.org Authentication-Results: ozlabs.org; dkim=pass (2048-bit key; secure) header.d=lists.infradead.org header.i=@lists.infradead.org header.a=rsa-sha256 header.s=merlin.20170209 header.b=WPfZsNlQ; dkim-atps=neutral Received: from merlin.infradead.org (merlin.infradead.org [IPv6:2001:8b0:10b:1231::1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 4Bb71z3gjdz9sSP for ; Tue, 25 Aug 2020 09:02:47 +1000 (AEST) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=merlin.20170209; h=Sender:Content-Transfer-Encoding: Content-Type:Cc:List-Subscribe:List-Help:List-Post:List-Archive: List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To:Message-Id:Date: Subject:To:From:Reply-To:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=SIUJ5XvRw6IEg18/+LADCxQW2A12fzOs3/n2Tw3QIi4=; b=WPfZsNlQUUNJd23ADKqu37Ocq qFoBxTFrUp9QE3je7M3tWj64rA80inCRNBfr9d+5CVs7IvQlB8/lxCNBEDZM78SjFtAb3XQh6tbmI DTysRhf1GZpFVP+oKkvOF8+obSIyTUKLEHa4XW6KOrxOFx0NT/J6Tfx9xnNK2JJE4Jg2uBIE0fvPp OF9q83VBLmhBxtij7Q5aY0m36vUm5TwLVwZf6OflaJ9tU27QrSklePziCRVZ6BZmv3BDtFEyCTu31 HG8DNRgNislBcJ5vhrXYVQmtBO1ckuCB2naGberFQElbrrJQ0fvkZobzZY811I2hrf63U9iH8c9Lq VCvSPU7Fg==; Received: from localhost ([::1] helo=merlin.infradead.org) by merlin.infradead.org with esmtp (Exim 4.92.3 #3 (Red Hat Linux)) id 1kALSS-0008MJ-HH; Mon, 24 Aug 2020 23:00:56 +0000 Received: from mails.bitsofnetworks.org ([2001:912:1800:ff::131]) by merlin.infradead.org with esmtps (Exim 4.92.3 #3 (Red Hat Linux)) id 1kALSK-0008Ih-Nh for openwrt-devel@lists.openwrt.org; Mon, 24 Aug 2020 23:00:51 +0000 Received: from [2001:912:1800::5c8] (helo=localhost.localdomain) by mails.bitsofnetworks.org with esmtp (Exim 4.89) (envelope-from ) id 1kALSG-000419-Pa; Tue, 25 Aug 2020 01:00:44 +0200 From: Baptiste Jonglez To: openwrt-devel@lists.openwrt.org Subject: [PATCH opkg 1/5] download: remove compatibility with old cache naming scheme Date: Tue, 25 Aug 2020 01:00:28 +0200 Message-Id: <20200824230032.492951-2-baptiste@bitsofnetworks.org> X-Mailer: git-send-email 2.27.0 In-Reply-To: <20200824230032.492951-1-baptiste@bitsofnetworks.org> References: <20200824230032.492951-1-baptiste@bitsofnetworks.org> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20200824_190049_276778_56CDA38A X-CRM114-Status: GOOD ( 15.61 ) X-Spam-Score: 0.0 (/) X-Spam-Report: SpamAssassin version 3.4.4 on merlin.infradead.org summary: Content analysis details: (0.0 points) pts rule name description ---- ---------------------- -------------------------------------------------- 0.0 SPF_HELO_NONE SPF: HELO does not publish an SPF Record -0.0 SPF_PASS SPF: sender matches SPF record X-BeenThere: openwrt-devel@lists.openwrt.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: OpenWrt Development List List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Baptiste Jonglez Sender: "openwrt-devel" Errors-To: openwrt-devel-bounces+incoming=patchwork.ozlabs.org@lists.openwrt.org From: Baptiste Jonglez A long time ago, the name of cached files was derived from the source URL. This was changed in 2011 with d46db43e21 ("Don't include the source URI in the cached filename.") Some compatibility code was left behind: even today, we are still trying to read from the old filename. The goal of this compatibility code was to account for existing caches that still had files with the old naming scheme. More than 9 years later, it is safe to remove this compatibility code. It simplifies the download code and avoids a useless disk access. Signed-off-by: Baptiste Jonglez --- libopkg/opkg_download.c | 36 +++++++++++------------------------- 1 file changed, 11 insertions(+), 25 deletions(-) diff --git a/libopkg/opkg_download.c b/libopkg/opkg_download.c index e970506..97df37f 100644 --- a/libopkg/opkg_download.c +++ b/libopkg/opkg_download.c @@ -131,8 +131,7 @@ opkg_download(const char *src, const char *dest_file_name, static int opkg_download_cache(const char *src, const char *dest_file_name) { - char *cache_name = xstrdup(src); - char *cache_location, *p; + char *cache_name, *cache_location; int err = 0; if (!conf->cache || str_starts_with(src, "file:")) { @@ -146,32 +145,19 @@ opkg_download_cache(const char *src, const char *dest_file_name) goto out1; } - for (p = cache_name; *p; p++) - if (*p == '/') - *p = ','; /* looks nicer than | or # */ - + char *filename = strrchr(dest_file_name, '/'); + if (filename) + cache_name = xstrdup(filename + 1); // strip leading '/' + else + cache_name = xstrdup(dest_file_name); sprintf_alloc(&cache_location, "%s/%s", conf->cache, cache_name); if (file_exists(cache_location)) opkg_msg(NOTICE, "Copying %s.\n", cache_location); else { - /* cache file with funky name not found, try simple name */ - free(cache_name); - char *filename = strrchr(dest_file_name, '/'); - if (filename) - cache_name = xstrdup(filename + 1); // strip leading '/' - else - cache_name = xstrdup(dest_file_name); - free(cache_location); - sprintf_alloc(&cache_location, "%s/%s", conf->cache, - cache_name); - if (file_exists(cache_location)) - opkg_msg(NOTICE, "Copying %s.\n", cache_location); - else { - err = opkg_download(src, cache_location, 0); - if (err) { - (void)unlink(cache_location); - goto out2; - } + err = opkg_download(src, cache_location, 0); + if (err) { + (void)unlink(cache_location); + goto out2; } } @@ -179,8 +165,8 @@ opkg_download_cache(const char *src, const char *dest_file_name) out2: free(cache_location); -out1: free(cache_name); +out1: return err; } From patchwork Mon Aug 24 23:00:29 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Baptiste Jonglez X-Patchwork-Id: 1350730 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=lists.openwrt.org (client-ip=2001:8b0:10b:1231::1; helo=merlin.infradead.org; envelope-from=openwrt-devel-bounces+incoming=patchwork.ozlabs.org@lists.openwrt.org; receiver=) Authentication-Results: ozlabs.org; dmarc=none (p=none dis=none) header.from=bitsofnetworks.org Authentication-Results: ozlabs.org; dkim=pass (2048-bit key; secure) header.d=lists.infradead.org header.i=@lists.infradead.org header.a=rsa-sha256 header.s=merlin.20170209 header.b=oJyJQ0hM; dkim-atps=neutral Received: from merlin.infradead.org (merlin.infradead.org [IPv6:2001:8b0:10b:1231::1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 4Bb71z5MFXz9sTR for ; Tue, 25 Aug 2020 09:02:47 +1000 (AEST) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=merlin.20170209; h=Sender:Content-Transfer-Encoding: Content-Type:Cc:List-Subscribe:List-Help:List-Post:List-Archive: List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To:Message-Id:Date: Subject:To:From:Reply-To:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=Sk8Gi9xmWD4S04upt8rh1P5/0a/4oyOxfJHJ8EosXJg=; b=oJyJQ0hMkOderkWY//tWJFvgS oJEVA2/G7HUwwRoBFz/FNX6mNwOC1B6BmwA5V7pDzD2Qvmf1ZzEtskrJCdZcBm/7BA0rRCrDYoXcU UZjpsCLR0pyrokJYtcQuVdUD3ZB5mZHxLU0tUmmi3WS8yk5QGAfxEabFl9bHja8rx7qlPHwfZiH03 CuWOMdU5HEqA3vNgtjsfI9WEpVX3HV4iw0sB2vHUDv5noKe0PlNg6xOptSbuKFFvX2Qney/KquKbv KmOgzq26VttL74sCaaYzhEk0DWCUo8vzO4skAiP/3TPok8Jf+tW+vSVjjAszM4dOwFNivP6qzoQTt geDtIfK1w==; Received: from localhost ([::1] helo=merlin.infradead.org) by merlin.infradead.org with esmtp (Exim 4.92.3 #3 (Red Hat Linux)) id 1kALSY-0008O0-Ba; Mon, 24 Aug 2020 23:01:02 +0000 Received: from mails.bitsofnetworks.org ([2001:912:1800:ff::131]) by merlin.infradead.org with esmtps (Exim 4.92.3 #3 (Red Hat Linux)) id 1kALSK-0008Id-Ng for openwrt-devel@lists.openwrt.org; Mon, 24 Aug 2020 23:00:51 +0000 Received: from [2001:912:1800::5c8] (helo=localhost.localdomain) by mails.bitsofnetworks.org with esmtp (Exim 4.89) (envelope-from ) id 1kALSG-000419-SE; Tue, 25 Aug 2020 01:00:44 +0200 From: Baptiste Jonglez To: openwrt-devel@lists.openwrt.org Subject: [PATCH opkg 2/5] libopkg: factor out checksum and size verification Date: Tue, 25 Aug 2020 01:00:29 +0200 Message-Id: <20200824230032.492951-3-baptiste@bitsofnetworks.org> X-Mailer: git-send-email 2.27.0 In-Reply-To: <20200824230032.492951-1-baptiste@bitsofnetworks.org> References: <20200824230032.492951-1-baptiste@bitsofnetworks.org> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20200824_190049_111173_6866E563 X-CRM114-Status: GOOD ( 19.32 ) X-Spam-Score: 0.0 (/) X-Spam-Report: SpamAssassin version 3.4.4 on merlin.infradead.org summary: Content analysis details: (0.0 points) pts rule name description ---- ---------------------- -------------------------------------------------- 0.0 SPF_HELO_NONE SPF: HELO does not publish an SPF Record -0.0 SPF_PASS SPF: sender matches SPF record X-BeenThere: openwrt-devel@lists.openwrt.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: OpenWrt Development List List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Baptiste Jonglez Sender: "openwrt-devel" Errors-To: openwrt-devel-bounces+incoming=patchwork.ozlabs.org@lists.openwrt.org From: Baptiste Jonglez This is a sizeable chunk of code that be can pretty well isolated in its own function. This refactoring will be necessary for an upcoming feature in which opkg_download_pkg() will need to verify the checksum of packages in the cache. This is the reason why the new function is located in `opkg_download.c`. Signed-off-by: Baptiste Jonglez --- libopkg/opkg_download.c | 78 +++++++++++++++++++++++++++++++++++++++++ libopkg/opkg_download.h | 1 + libopkg/opkg_install.c | 71 ++----------------------------------- 3 files changed, 81 insertions(+), 69 deletions(-) diff --git a/libopkg/opkg_download.c b/libopkg/opkg_download.c index 97df37f..f7c2f88 100644 --- a/libopkg/opkg_download.c +++ b/libopkg/opkg_download.c @@ -35,6 +35,84 @@ static int str_starts_with(const char *str, const char *prefix) return (strncmp(str, prefix, strlen(prefix)) == 0); } +int opkg_verify_integrity(pkg_t *pkg, const char *filename) +{ + int err = 0; + char *file_md5, *pkg_md5; + char *file_sha256, *pkg_sha256; + struct stat pkg_stat; + long long int pkg_expected_size; + + /* Check file size */ + err = lstat(filename, &pkg_stat); + + if (err) { + opkg_msg(ERROR, "Failed to stat %s: %s\n", + filename, strerror(errno)); + return err; + } + + pkg_expected_size = pkg_get_int(pkg, PKG_SIZE); + + if (pkg_expected_size > 0 && pkg_stat.st_size != pkg_expected_size) { + if (!conf->force_checksum) { + opkg_msg(ERROR, + "Package size mismatch: %s is %lld bytes, expecting %lld bytes\n", + pkg->name, (long long int)pkg_stat.st_size, pkg_expected_size); + return -1; + } else { + opkg_msg(NOTICE, + "Ignored %s size mismatch.\n", + pkg->name); + } + } + + /* Check for md5 values */ + pkg_md5 = pkg_get_md5(pkg); + if (pkg_md5) { + file_md5 = file_md5sum_alloc(filename); + if (file_md5 && strcmp(file_md5, pkg_md5)) { + if (!conf->force_checksum) { + opkg_msg(ERROR, "Package %s md5sum mismatch. " + "Either the opkg or the package index are corrupt. " + "Try 'opkg update'.\n", pkg->name); + free(file_md5); + return -1; + } else { + opkg_msg(NOTICE, + "Ignored %s md5sum mismatch.\n", + pkg->name); + } + } + if (file_md5) + free(file_md5); + } + + /* Check for sha256 value */ + pkg_sha256 = pkg_get_sha256(pkg); + if (pkg_sha256) { + file_sha256 = file_sha256sum_alloc(filename); + if (file_sha256 && strcmp(file_sha256, pkg_sha256)) { + if (!conf->force_checksum) { + opkg_msg(ERROR, + "Package %s sha256sum mismatch. " + "Either the opkg or the package index are corrupt. " + "Try 'opkg update'.\n", pkg->name); + free(file_sha256); + return -1; + } else { + opkg_msg(NOTICE, + "Ignored %s sha256sum mismatch.\n", + pkg->name); + } + } + if (file_sha256) + free(file_sha256); + } + + return err; +} + int opkg_download(const char *src, const char *dest_file_name, const short hide_error) diff --git a/libopkg/opkg_download.h b/libopkg/opkg_download.h index fd260bc..863bcf2 100644 --- a/libopkg/opkg_download.h +++ b/libopkg/opkg_download.h @@ -20,6 +20,7 @@ #include "pkg.h" +int opkg_verify_integrity(pkg_t *pkg, const char *filename); int opkg_download(const char *src, const char *dest_file_name, const short hide_error); int opkg_download_pkg(pkg_t * pkg, const char *dir); diff --git a/libopkg/opkg_install.c b/libopkg/opkg_install.c index 27c9484..e8eccf8 100644 --- a/libopkg/opkg_install.c +++ b/libopkg/opkg_install.c @@ -1250,12 +1250,8 @@ int opkg_install_pkg(pkg_t * pkg, int from_upgrade) pkg_vec_t *replacees; abstract_pkg_t *ab_pkg = NULL; int old_state_flag; - char *file_md5, *pkg_md5; - char *file_sha256, *pkg_sha256; sigset_t newset, oldset; const char *local_filename; - long long int pkg_expected_size; - struct stat pkg_stat; time_t now; if (from_upgrade) @@ -1367,72 +1363,9 @@ int opkg_install_pkg(pkg_t * pkg, int from_upgrade) } #endif - /* Check file size */ - err = lstat(local_filename, &pkg_stat); - - if (err) { - opkg_msg(ERROR, "Failed to stat %s: %s\n", - local_filename, strerror(errno)); + err = opkg_verify_integrity(pkg, local_filename); + if (err) return -1; - } - - pkg_expected_size = pkg_get_int(pkg, PKG_SIZE); - - if (pkg_expected_size > 0 && pkg_stat.st_size != pkg_expected_size) { - if (!conf->force_checksum) { - opkg_msg(ERROR, - "Package size mismatch: %s is %lld bytes, expecting %lld bytes\n", - pkg->name, (long long int)pkg_stat.st_size, pkg_expected_size); - return -1; - } else { - opkg_msg(NOTICE, - "Ignored %s size mismatch.\n", - pkg->name); - } - } - - /* Check for md5 values */ - pkg_md5 = pkg_get_md5(pkg); - if (pkg_md5) { - file_md5 = file_md5sum_alloc(local_filename); - if (file_md5 && strcmp(file_md5, pkg_md5)) { - if (!conf->force_checksum) { - opkg_msg(ERROR, "Package %s md5sum mismatch. " - "Either the opkg or the package index are corrupt. " - "Try 'opkg update'.\n", pkg->name); - free(file_md5); - return -1; - } else { - opkg_msg(NOTICE, - "Ignored %s md5sum mismatch.\n", - pkg->name); - } - } - if (file_md5) - free(file_md5); - } - - /* Check for sha256 value */ - pkg_sha256 = pkg_get_sha256(pkg); - if (pkg_sha256) { - file_sha256 = file_sha256sum_alloc(local_filename); - if (file_sha256 && strcmp(file_sha256, pkg_sha256)) { - if (!conf->force_checksum) { - opkg_msg(ERROR, - "Package %s sha256sum mismatch. " - "Either the opkg or the package index are corrupt. " - "Try 'opkg update'.\n", pkg->name); - free(file_sha256); - return -1; - } else { - opkg_msg(NOTICE, - "Ignored %s sha256sum mismatch.\n", - pkg->name); - } - } - if (file_sha256) - free(file_sha256); - } if (conf->download_only) { if (conf->nodeps == 0) { From patchwork Mon Aug 24 23:00:30 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Baptiste Jonglez X-Patchwork-Id: 1350731 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=lists.openwrt.org (client-ip=2001:8b0:10b:1231::1; helo=merlin.infradead.org; envelope-from=openwrt-devel-bounces+incoming=patchwork.ozlabs.org@lists.openwrt.org; receiver=) Authentication-Results: ozlabs.org; dmarc=none (p=none dis=none) header.from=bitsofnetworks.org Authentication-Results: ozlabs.org; dkim=pass (2048-bit key; secure) header.d=lists.infradead.org header.i=@lists.infradead.org header.a=rsa-sha256 header.s=merlin.20170209 header.b=i+Rq7aiN; dkim-atps=neutral Received: from merlin.infradead.org (merlin.infradead.org [IPv6:2001:8b0:10b:1231::1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 4Bb71z4cR1z9sTN for ; Tue, 25 Aug 2020 09:02:47 +1000 (AEST) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=merlin.20170209; h=Sender:Content-Transfer-Encoding: Content-Type:Cc:List-Subscribe:List-Help:List-Post:List-Archive: List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To:Message-Id:Date: Subject:To:From:Reply-To:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=HlQE3oo8XQy2lyijF4AePNEkJ0QKqdTjlfznxFS+L+0=; b=i+Rq7aiNe6tjsDCkYB/VcSnHz nbZnU9eeIlSSMnhg7sUXjYJD6AKP9OOEfQnH/WAjfT67nrsU862E2o/tMu+TbA23NEkuSecPm0zVz ryKkCd1Wm2KE2wtravWFFr/MPOBDmO/g+21k344jg1KM+ZsETjdy5YTDLonjNrfDPOcteFuHKPUSa mZJA33i/wDXFkfhLtfrCHsTWkIjs8KV0oOEWCUTIml6kUo6SdCOvYwL9+xLb5nFeJTt0gY/PKWb1h zCnLLxH3dWm+ykRwMBk3kTMJKS6wKd0sBnAItktiPpXub2J4/fAh5rRAiKDKrgvavrM1mKKwjkjPm 139VrGoyg==; Received: from localhost ([::1] helo=merlin.infradead.org) by merlin.infradead.org with esmtp (Exim 4.92.3 #3 (Red Hat Linux)) id 1kALSQ-0008Lc-2A; Mon, 24 Aug 2020 23:00:54 +0000 Received: from mails.bitsofnetworks.org ([2001:912:1800:ff::131]) by merlin.infradead.org with esmtps (Exim 4.92.3 #3 (Red Hat Linux)) id 1kALSK-0008Ig-Ng for openwrt-devel@lists.openwrt.org; Mon, 24 Aug 2020 23:00:50 +0000 Received: from [2001:912:1800::5c8] (helo=localhost.localdomain) by mails.bitsofnetworks.org with esmtp (Exim 4.89) (envelope-from ) id 1kALSG-000419-Us; Tue, 25 Aug 2020 01:00:44 +0200 From: Baptiste Jonglez To: openwrt-devel@lists.openwrt.org Subject: [PATCH opkg 3/5] download: factor out the logic for building cache filenames Date: Tue, 25 Aug 2020 01:00:30 +0200 Message-Id: <20200824230032.492951-4-baptiste@bitsofnetworks.org> X-Mailer: git-send-email 2.27.0 In-Reply-To: <20200824230032.492951-1-baptiste@bitsofnetworks.org> References: <20200824230032.492951-1-baptiste@bitsofnetworks.org> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20200824_190049_004258_47C35780 X-CRM114-Status: GOOD ( 12.31 ) X-Spam-Score: 0.0 (/) X-Spam-Report: SpamAssassin version 3.4.4 on merlin.infradead.org summary: Content analysis details: (0.0 points) pts rule name description ---- ---------------------- -------------------------------------------------- 0.0 SPF_HELO_NONE SPF: HELO does not publish an SPF Record -0.0 SPF_PASS SPF: sender matches SPF record X-BeenThere: openwrt-devel@lists.openwrt.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: OpenWrt Development List List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Baptiste Jonglez Sender: "openwrt-devel" Errors-To: openwrt-devel-bounces+incoming=patchwork.ozlabs.org@lists.openwrt.org From: Baptiste Jonglez If we want to access files in the cache from multiple functions, it is necessary to have a single source of truth regarding the naming of files in the cache. Signed-off-by: Baptiste Jonglez --- libopkg/opkg_download.c | 17 ++++++++++++----- 1 file changed, 12 insertions(+), 5 deletions(-) diff --git a/libopkg/opkg_download.c b/libopkg/opkg_download.c index f7c2f88..175282c 100644 --- a/libopkg/opkg_download.c +++ b/libopkg/opkg_download.c @@ -206,6 +206,17 @@ opkg_download(const char *src, const char *dest_file_name, return err; } +static char* get_cache_filename(const char *dest_file_name) +{ + char *cache_name; + char *filename = strrchr(dest_file_name, '/'); + if (filename) + cache_name = xstrdup(filename + 1); // strip leading '/' + else + cache_name = xstrdup(dest_file_name); + return cache_name; +} + static int opkg_download_cache(const char *src, const char *dest_file_name) { @@ -223,11 +234,7 @@ opkg_download_cache(const char *src, const char *dest_file_name) goto out1; } - char *filename = strrchr(dest_file_name, '/'); - if (filename) - cache_name = xstrdup(filename + 1); // strip leading '/' - else - cache_name = xstrdup(dest_file_name); + cache_name = get_cache_filename(dest_file_name); sprintf_alloc(&cache_location, "%s/%s", conf->cache, cache_name); if (file_exists(cache_location)) opkg_msg(NOTICE, "Copying %s.\n", cache_location); From patchwork Mon Aug 24 23:00:31 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Baptiste Jonglez X-Patchwork-Id: 1350732 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=lists.openwrt.org (client-ip=2001:8b0:10b:1231::1; helo=merlin.infradead.org; envelope-from=openwrt-devel-bounces+incoming=patchwork.ozlabs.org@lists.openwrt.org; receiver=) Authentication-Results: ozlabs.org; dmarc=none (p=none dis=none) header.from=bitsofnetworks.org Authentication-Results: ozlabs.org; dkim=pass (2048-bit key; secure) header.d=lists.infradead.org header.i=@lists.infradead.org header.a=rsa-sha256 header.s=merlin.20170209 header.b=l6NQO/mQ; dkim-atps=neutral Received: from merlin.infradead.org (merlin.infradead.org [IPv6:2001:8b0:10b:1231::1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 4Bb7203wv7z9sTT for ; Tue, 25 Aug 2020 09:02:48 +1000 (AEST) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=merlin.20170209; h=Sender:Content-Transfer-Encoding: Content-Type:Cc:List-Subscribe:List-Help:List-Post:List-Archive: List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To:Message-Id:Date: Subject:To:From:Reply-To:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=yEDcUJiJwOlCXcD29VcxkTAXuZokyOM3KnNPPfyrrKs=; b=l6NQO/mQU1qdpZdXgkio9zd6p d7b5SklWuOUgR7HyK33FQIIs1jSgJ4xlASn5AuvJ2YXbMtameqaJVS0896E6DdASeh3YB5wiVSyzl LfswpcrJL0EuMNdPGkLnd8txp14bdbZm8C1kW2C216F8nnX8YJbwVeOX8mvsus4CmnRN2jbI6W24F cDZof2GXguIff9+nubcngANTF4+c4hUmUzuH+iYGtNq4nePZqSbms5D91ZfwezxwVd6wv5EYhU45u eLcQWIdsrGsxwiPEob1jI2ZDSgcj2ylcqlWDywyYDGONlhho81ozkI9na6hjCf1roIS30bFMLtbhA UFMjAukkw==; Received: from localhost ([::1] helo=merlin.infradead.org) by merlin.infradead.org with esmtp (Exim 4.92.3 #3 (Red Hat Linux)) id 1kALSV-0008Mm-9j; Mon, 24 Aug 2020 23:00:59 +0000 Received: from mails.bitsofnetworks.org ([2001:912:1800:ff::131]) by merlin.infradead.org with esmtps (Exim 4.92.3 #3 (Red Hat Linux)) id 1kALSK-0008Ii-Ni for openwrt-devel@lists.openwrt.org; Mon, 24 Aug 2020 23:00:51 +0000 Received: from [2001:912:1800::5c8] (helo=localhost.localdomain) by mails.bitsofnetworks.org with esmtp (Exim 4.89) (envelope-from ) id 1kALSG-000419-Vl; Tue, 25 Aug 2020 01:00:44 +0200 From: Baptiste Jonglez To: openwrt-devel@lists.openwrt.org Subject: [PATCH opkg 4/5] download: purge cached packages that have incorrect checksum Date: Tue, 25 Aug 2020 01:00:31 +0200 Message-Id: <20200824230032.492951-5-baptiste@bitsofnetworks.org> X-Mailer: git-send-email 2.27.0 In-Reply-To: <20200824230032.492951-1-baptiste@bitsofnetworks.org> References: <20200824230032.492951-1-baptiste@bitsofnetworks.org> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20200824_190049_261577_2667C793 X-CRM114-Status: GOOD ( 15.37 ) X-Spam-Score: 0.0 (/) X-Spam-Report: SpamAssassin version 3.4.4 on merlin.infradead.org summary: Content analysis details: (0.0 points) pts rule name description ---- ---------------------- -------------------------------------------------- 0.0 SPF_HELO_NONE SPF: HELO does not publish an SPF Record -0.0 SPF_PASS SPF: sender matches SPF record X-BeenThere: openwrt-devel@lists.openwrt.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: OpenWrt Development List List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Baptiste Jonglez Sender: "openwrt-devel" Errors-To: openwrt-devel-bounces+incoming=patchwork.ozlabs.org@lists.openwrt.org From: Baptiste Jonglez Before using a package from the cache, verify its size and checksum against a package index, and delete the package from the cache if they don't match. The install process will then proceed to download the "fixed" package as usual. This allows to cope with remote packages that are rebuilt while keeping the same version number as packages in the local cache. With this change, any outdated package in the local cache will be purged and the new version will be downloaded instead. This is mostly useful when running opkg on the host (e.g. in the imagebuilder). When running on a device, no cache is configured by default, so this change does nothing in that case. Fixes: FS#2690 Signed-off-by: Baptiste Jonglez --- libopkg/opkg_download.c | 19 +++++++++++++++++++ 1 file changed, 19 insertions(+) diff --git a/libopkg/opkg_download.c b/libopkg/opkg_download.c index 175282c..3b79856 100644 --- a/libopkg/opkg_download.c +++ b/libopkg/opkg_download.c @@ -263,6 +263,8 @@ int opkg_download_pkg(pkg_t * pkg, const char *dir) char *stripped_filename; char *urlencoded_path; char *filename; + char *cache_name; + char *cache_location; if (pkg->src == NULL) { opkg_msg(ERROR, @@ -296,6 +298,23 @@ int opkg_download_pkg(pkg_t * pkg, const char *dir) sprintf_alloc(&local_filename, "%s/%s", dir, stripped_filename); pkg_set_string(pkg, PKG_LOCAL_FILENAME, local_filename); + /* Invalidate/remove cached package if it has an incorrect checksum. */ + if (conf->cache) { + cache_name = get_cache_filename(local_filename); + sprintf_alloc(&cache_location, "%s/%s", conf->cache, cache_name); + free(cache_name); + if (file_exists(cache_location)) { + err = opkg_verify_integrity(pkg, cache_location); + if (err) { + opkg_msg(NOTICE, + "Removing %s from cache because it has incorrect checksum.\n", + pkg->name); + unlink(cache_location); + } + } + free(cache_location); + } + err = opkg_download_cache(url, local_filename); free(url); From patchwork Mon Aug 24 23:00:32 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Baptiste Jonglez X-Patchwork-Id: 1350734 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=lists.openwrt.org (client-ip=2001:8b0:10b:1231::1; helo=merlin.infradead.org; envelope-from=openwrt-devel-bounces+incoming=patchwork.ozlabs.org@lists.openwrt.org; receiver=) Authentication-Results: ozlabs.org; dmarc=none (p=none dis=none) header.from=bitsofnetworks.org Authentication-Results: ozlabs.org; dkim=pass (2048-bit key; secure) header.d=lists.infradead.org header.i=@lists.infradead.org header.a=rsa-sha256 header.s=merlin.20170209 header.b=kx4s3IzU; dkim-atps=neutral Received: from merlin.infradead.org (merlin.infradead.org [IPv6:2001:8b0:10b:1231::1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 4Bb7231jJ8z9sSP for ; Tue, 25 Aug 2020 09:02:50 +1000 (AEST) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=merlin.20170209; h=Sender:Content-Transfer-Encoding: Content-Type:Cc:List-Subscribe:List-Help:List-Post:List-Archive: List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To:Message-Id:Date: Subject:To:From:Reply-To:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=tQzDiF5p0nKRc4UzSmZrkP2W1nHdaVmZMlUCGbujyUs=; b=kx4s3IzUoyO5zFUm0l89N3tyy GV7OUx1DVv3OIImwlyDjz94YBkke17ETAZpShd62lRKV4W/NRS7ld2+HUt47FI9Wbq/4IGM15PYqn jTLJLq55tjfoVD5M8xZidCeiAIBl8M9Zxi8SHkaaMI7pKz/u4Q5sEacqvlEBQ9AOdX4zOLhd9y9kn AD4zN7MH4zAsluatWwCDndPFvFut/++DgJ5RflDMS13T2IcpYxIY9p/5OUUeFs+ggOMIdJTz44OY+ 2nhnotknp9TmjupogU/86H2C6KV4y0vp+zz7DPa9KJSgPnHt4NqkDDPY91/itAqfSQkAI4pL+c1d3 i55AEWl1A==; Received: from localhost ([::1] helo=merlin.infradead.org) by merlin.infradead.org with esmtp (Exim 4.92.3 #3 (Red Hat Linux)) id 1kALSR-0008Lw-DC; Mon, 24 Aug 2020 23:00:55 +0000 Received: from mails.bitsofnetworks.org ([2001:912:1800:ff::131]) by merlin.infradead.org with esmtps (Exim 4.92.3 #3 (Red Hat Linux)) id 1kALSK-0008If-Nh for openwrt-devel@lists.openwrt.org; Mon, 24 Aug 2020 23:00:51 +0000 Received: from [2001:912:1800::5c8] (helo=localhost.localdomain) by mails.bitsofnetworks.org with esmtp (Exim 4.89) (envelope-from ) id 1kALSH-000419-0N; Tue, 25 Aug 2020 01:00:45 +0200 From: Baptiste Jonglez To: openwrt-devel@lists.openwrt.org Subject: [PATCH opkg 5/5] opkg_verify_integrity: better logging and error conditions Date: Tue, 25 Aug 2020 01:00:32 +0200 Message-Id: <20200824230032.492951-6-baptiste@bitsofnetworks.org> X-Mailer: git-send-email 2.27.0 In-Reply-To: <20200824230032.492951-1-baptiste@bitsofnetworks.org> References: <20200824230032.492951-1-baptiste@bitsofnetworks.org> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20200824_190049_227188_414A8F75 X-CRM114-Status: GOOD ( 16.25 ) X-Spam-Score: 0.0 (/) X-Spam-Report: SpamAssassin version 3.4.4 on merlin.infradead.org summary: Content analysis details: (0.0 points) pts rule name description ---- ---------------------- -------------------------------------------------- 0.0 SPF_HELO_NONE SPF: HELO does not publish an SPF Record -0.0 SPF_PASS SPF: sender matches SPF record X-BeenThere: openwrt-devel@lists.openwrt.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: OpenWrt Development List List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Baptiste Jonglez Sender: "openwrt-devel" Errors-To: openwrt-devel-bounces+incoming=patchwork.ozlabs.org@lists.openwrt.org From: Baptiste Jonglez The function now always returns an error if size/checksum don't match: we let the caller decide what to do with the result. In addition, most of the logging is also moved to the caller. We just keep logging for unexpected errors and a bit of debug at loglevel INFO. Signed-off-by: Baptiste Jonglez --- libopkg/opkg_download.c | 49 ++++++++++++++--------------------------- libopkg/opkg_install.c | 11 ++++++++- 2 files changed, 26 insertions(+), 34 deletions(-) diff --git a/libopkg/opkg_download.c b/libopkg/opkg_download.c index 3b79856..a9ebd58 100644 --- a/libopkg/opkg_download.c +++ b/libopkg/opkg_download.c @@ -55,16 +55,11 @@ int opkg_verify_integrity(pkg_t *pkg, const char *filename) pkg_expected_size = pkg_get_int(pkg, PKG_SIZE); if (pkg_expected_size > 0 && pkg_stat.st_size != pkg_expected_size) { - if (!conf->force_checksum) { - opkg_msg(ERROR, - "Package size mismatch: %s is %lld bytes, expecting %lld bytes\n", - pkg->name, (long long int)pkg_stat.st_size, pkg_expected_size); - return -1; - } else { - opkg_msg(NOTICE, - "Ignored %s size mismatch.\n", - pkg->name); - } + opkg_msg(INFO, + "Package size mismatch: %s is %lld bytes, expecting %lld bytes\n", + pkg->name, (long long int)pkg_stat.st_size, pkg_expected_size); + err = -1; + goto out; } /* Check for md5 values */ @@ -72,17 +67,11 @@ int opkg_verify_integrity(pkg_t *pkg, const char *filename) if (pkg_md5) { file_md5 = file_md5sum_alloc(filename); if (file_md5 && strcmp(file_md5, pkg_md5)) { - if (!conf->force_checksum) { - opkg_msg(ERROR, "Package %s md5sum mismatch. " - "Either the opkg or the package index are corrupt. " - "Try 'opkg update'.\n", pkg->name); - free(file_md5); - return -1; - } else { - opkg_msg(NOTICE, - "Ignored %s md5sum mismatch.\n", - pkg->name); - } + opkg_msg(INFO, "Package %s md5sum mismatch.\n", + pkg->name); + err = -1; + free(file_md5); + goto out; } if (file_md5) free(file_md5); @@ -93,23 +82,17 @@ int opkg_verify_integrity(pkg_t *pkg, const char *filename) if (pkg_sha256) { file_sha256 = file_sha256sum_alloc(filename); if (file_sha256 && strcmp(file_sha256, pkg_sha256)) { - if (!conf->force_checksum) { - opkg_msg(ERROR, - "Package %s sha256sum mismatch. " - "Either the opkg or the package index are corrupt. " - "Try 'opkg update'.\n", pkg->name); - free(file_sha256); - return -1; - } else { - opkg_msg(NOTICE, - "Ignored %s sha256sum mismatch.\n", - pkg->name); - } + opkg_msg(INFO, "Package %s sha256sum mismatch.\n", + pkg->name); + err = -1; + free(file_sha256); + goto out; } if (file_sha256) free(file_sha256); } +out: return err; } diff --git a/libopkg/opkg_install.c b/libopkg/opkg_install.c index e8eccf8..da55b48 100644 --- a/libopkg/opkg_install.c +++ b/libopkg/opkg_install.c @@ -1363,9 +1363,18 @@ int opkg_install_pkg(pkg_t * pkg, int from_upgrade) } #endif + /* Check integrity (size, checksums) */ err = opkg_verify_integrity(pkg, local_filename); - if (err) + if (err && !conf->force_checksum) { + opkg_msg(ERROR, "Checksum or size mismatch for package %s. " + "Either the opkg or the package index are corrupt. " + "Try 'opkg update'.\n", pkg->name); return -1; + } + if (err && conf->force_checksum) { + opkg_msg(NOTICE, "Ignored %s checksum or size mismatch.\n", + pkg->name); + } if (conf->download_only) { if (conf->nodeps == 0) {