From patchwork Mon Aug 24 23:00:27 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Baptiste Jonglez X-Patchwork-Id: 1350733 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=lists.openwrt.org (client-ip=2001:8b0:10b:1231::1; helo=merlin.infradead.org; envelope-from=openwrt-devel-bounces+incoming=patchwork.ozlabs.org@lists.openwrt.org; receiver=) Authentication-Results: ozlabs.org; dmarc=none (p=none dis=none) header.from=bitsofnetworks.org Authentication-Results: ozlabs.org; dkim=pass (2048-bit key; secure) header.d=lists.infradead.org header.i=@lists.infradead.org header.a=rsa-sha256 header.s=merlin.20170209 header.b=bCqDRRp5; dkim-atps=neutral Received: from merlin.infradead.org (merlin.infradead.org [IPv6:2001:8b0:10b:1231::1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 4Bb7225DVpz9sTX for ; Tue, 25 Aug 2020 09:02:50 +1000 (AEST) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=merlin.20170209; h=Sender:Content-Transfer-Encoding: Content-Type:Cc:List-Subscribe:List-Help:List-Post:List-Archive: List-Unsubscribe:List-Id:MIME-Version:Message-Id:Date:Subject:To:From: Reply-To:Content-ID:Content-Description:Resent-Date:Resent-From:Resent-Sender :Resent-To:Resent-Cc:Resent-Message-ID:In-Reply-To:References:List-Owner; bh=rJ5A2TVe16GXuKlLLMW3cPano84kmX3HCE8T6sqQ+4U=; b=bCqDRRp5Lbij3TbFx9s0ZAMl+d ctumkdpzCM+n7lNZhv9deCud3U8IFpQvepZMO9K8PlvU7wT3/eZRI2lqmzus1XtBK6hcAKP4HyCQ+ OrmjFSGEdGuSkIqD4xszuVQ1yrvRB5Uu6JaJZwI4TSbb6aA5ljdWfihvKAYwYwBc8oB8fU+aFaEba Ie47krWZOAK0PK3hBWXFMi+2GFpBeuzYtD+CbOTsRkTuu78dR8JU6VOIdeGF/VL9l6QYIhBbHnRZK 2iVJtH0b69Be/FB3jZqj0B0WXfRJK/5VbbZ56fb0CB7xXptUQb1Yr3KAqVEIRsOZwsKKzOzxQcN8k B5Ubfriw==; Received: from localhost ([::1] helo=merlin.infradead.org) by merlin.infradead.org with esmtp (Exim 4.92.3 #3 (Red Hat Linux)) id 1kALSO-0008LE-Gg; Mon, 24 Aug 2020 23:00:52 +0000 Received: from mails.bitsofnetworks.org ([2001:912:1800:ff::131]) by merlin.infradead.org with esmtps (Exim 4.92.3 #3 (Red Hat Linux)) id 1kALSK-0008Ie-Ni for openwrt-devel@lists.openwrt.org; Mon, 24 Aug 2020 23:00:50 +0000 Received: from [2001:912:1800::5c8] (helo=localhost.localdomain) by mails.bitsofnetworks.org with esmtp (Exim 4.89) (envelope-from ) id 1kALSG-000419-NB; Tue, 25 Aug 2020 01:00:44 +0200 From: Baptiste Jonglez To: openwrt-devel@lists.openwrt.org Subject: [PATCH opkg 0/5] Purge packages from cache when they have incorrect checksum Date: Tue, 25 Aug 2020 01:00:27 +0200 Message-Id: <20200824230032.492951-1-baptiste@bitsofnetworks.org> X-Mailer: git-send-email 2.27.0 MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20200824_190048_963336_8106FE45 X-CRM114-Status: GOOD ( 11.37 ) X-Spam-Score: 0.0 (/) X-Spam-Report: SpamAssassin version 3.4.4 on merlin.infradead.org summary: Content analysis details: (0.0 points) pts rule name description ---- ---------------------- -------------------------------------------------- 0.0 SPF_HELO_NONE SPF: HELO does not publish an SPF Record -0.0 SPF_PASS SPF: sender matches SPF record X-BeenThere: openwrt-devel@lists.openwrt.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: OpenWrt Development List List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Baptiste Jonglez Sender: "openwrt-devel" Errors-To: openwrt-devel-bounces+incoming=patchwork.ozlabs.org@lists.openwrt.org From: Baptiste Jonglez The motivation of this patch series is to fix FS#2690. Because packages are continuously rebuilt, the ImageBuilder ends up with old packages in its cache, and fails because checksums don't match with the new package index from the download server. The approach to solve this problem is the following. Before using a package from the cache, verify its size and checksum against the package index, and delete the package from the cache if they don't match. The install process will then proceed to download the "fixed" package as usual. The main patch is "download: purge cached packages that have incorrect checksum". The other ones are either cleanup or refactoring to prepare for the main change. Baptiste Jonglez (5): download: remove compatibility with old cache naming scheme libopkg: factor out checksum and size verification download: factor out the logic for building cache filenames download: purge cached packages that have incorrect checksum opkg_verify_integrity: better logging and error conditions libopkg/opkg_download.c | 123 ++++++++++++++++++++++++++++++++-------- libopkg/opkg_download.h | 1 + libopkg/opkg_install.c | 76 +++---------------------- 3 files changed, 108 insertions(+), 92 deletions(-) Acked-by: Paul Spooren