From patchwork Thu Nov 19 21:20:50 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Petr_=C5=A0tetiar?= X-Patchwork-Id: 1403273 X-Patchwork-Delegate: ynezz@true.cz Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=lists.openwrt.org (client-ip=2001:8b0:10b:1231::1; helo=merlin.infradead.org; envelope-from=openwrt-devel-bounces+incoming=patchwork.ozlabs.org@lists.openwrt.org; receiver=) Authentication-Results: ozlabs.org; dmarc=none (p=none dis=none) header.from=true.cz Authentication-Results: ozlabs.org; dkim=pass (2048-bit key; secure) header.d=lists.infradead.org header.i=@lists.infradead.org header.a=rsa-sha256 header.s=merlin.20170209 header.b=LihWTn3y; dkim-atps=neutral Received: from merlin.infradead.org (merlin.infradead.org [IPv6:2001:8b0:10b:1231::1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 4CcXjJ2dpkz9sTR for ; Fri, 20 Nov 2020 08:23:32 +1100 (AEDT) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=merlin.20170209; h=Sender:Content-Transfer-Encoding: Content-Type:Cc:List-Subscribe:List-Help:List-Post:List-Archive: List-Unsubscribe:List-Id:MIME-Version:Message-Id:Date:Subject:To:From: Reply-To:Content-ID:Content-Description:Resent-Date:Resent-From:Resent-Sender :Resent-To:Resent-Cc:Resent-Message-ID:In-Reply-To:References:List-Owner; bh=HkA+dcsGcj09KELTxuo+/B1RpVO1VSxKQfL9UQajhOk=; b=LihWTn3yF2ASNA3qgeHiHTxesQ 71fGBkfO1mo60wZSKG4xIsF2YhgjIy8Uh7d2xf6OdGLxEvpVcsRY9rWyuxtqrL+ai+Qqtx6PHHb1w O7qIIR299dPlpnvTUnjNkpLJiVFk6nJuDDzSUEFMeZuzfcESLkCZJpcZlkOy5POABacrP9TZ4wrDZ r95M7jn860Ux5qhVgQXThTjM+4HXdd38bihWf2XXJL5Ku/KfNUQydxKFSevXOTe9WswmcZKJreGvR INQLauJaYjYpaV6DNUnnfrdzLH4doOQqpShsh1Eb5qozsn9SRs0LJMtYNUqq3paFIGtJ+AXnAtU4R fgu645Yw==; Received: from localhost ([::1] helo=merlin.infradead.org) by merlin.infradead.org with esmtp (Exim 4.92.3 #3 (Red Hat Linux)) id 1kfrMb-0002ep-Ef; Thu, 19 Nov 2020 21:21:09 +0000 Received: from smtp-out.xnet.cz ([178.217.244.18]) by merlin.infradead.org with esmtps (Exim 4.92.3 #3 (Red Hat Linux)) id 1kfrMW-0002bC-2U for openwrt-devel@lists.openwrt.org; Thu, 19 Nov 2020 21:21:06 +0000 Received: from meh.true.cz (meh.true.cz [108.61.167.218]) (Authenticated sender: petr@true.cz) by smtp-out.xnet.cz (Postfix) with ESMTPSA id B504132C2; Thu, 19 Nov 2020 22:20:58 +0100 (CET) Received: by meh.true.cz (OpenSMTPD) with ESMTP id 008f8b4b; Thu, 19 Nov 2020 22:20:36 +0100 (CET) From: =?utf-8?q?Petr_=C5=A0tetiar?= To: openwrt-devel@lists.openwrt.org Subject: [PATCH] download: handle possibly invalid local tarballs Date: Thu, 19 Nov 2020 22:20:50 +0100 Message-Id: <20201119212050.14005-1-ynezz@true.cz> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20201119_162104_276645_FFB60443 X-CRM114-Status: GOOD ( 18.56 ) X-Spam-Score: 0.0 (/) X-Spam-Report: SpamAssassin version 3.4.4 on merlin.infradead.org summary: Content analysis details: (0.0 points) pts rule name description ---- ---------------------- -------------------------------------------------- 0.0 SPF_NONE SPF: sender does not publish an SPF Record 0.0 SPF_HELO_NONE SPF: HELO does not publish an SPF Record X-BeenThere: openwrt-devel@lists.openwrt.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: OpenWrt Development List List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: =?utf-8?q?Petr_=C5=A0tetiar?= Sender: "openwrt-devel" Errors-To: openwrt-devel-bounces+incoming=patchwork.ozlabs.org@lists.openwrt.org Currently it's assumed, that already downloaded tarballs are always fine, so no checksum checking is performed and the tarball is used even if it might be corrupted. From now on, we're going to always check the downloaded tarballs before considering them valid. Steps to reproduce: 1. remove cached tarball rm dl/libubox-2020-08-06-9e52171d.tar.xz 2. download valid tarball again make package/libubox/download 3. invalidate the tarball sed -i 's/PKG_MIRROR_HASH:=../PKG_MIRROR_HASH:=ff/' package/libs/libubox/Makefile 4. now compile with corrupt tarball source make package/libubox/{clean,compile} Signed-off-by: Petr Štetiar --- include/download.mk | 2 +- scripts/download.pl | 18 ++++++++++++++++++ 2 files changed, 19 insertions(+), 1 deletion(-) diff --git a/include/download.mk b/include/download.mk index d393bf390716..7c2de929d347 100644 --- a/include/download.mk +++ b/include/download.mk @@ -317,7 +317,7 @@ define Download ) download: $(DL_DIR)/$(FILE) - $(DL_DIR)/$(FILE): + $(DL_DIR)/$(FILE): FORCE mkdir -p $(DL_DIR) $(call locked, \ $(if $(DownloadMethod/$(call dl_method,$(URL),$(PROTO))), \ diff --git a/scripts/download.pl b/scripts/download.pl index cdccae133f49..b51c8f1127e8 100755 --- a/scripts/download.pl +++ b/scripts/download.pl @@ -261,6 +261,24 @@ foreach my $mirror (@ARGV) { push @mirrors, 'https://sources.openwrt.org'; push @mirrors, 'https://mirror2.openwrt.org/sources'; +if (-f "$target/$filename") { + $hash_cmd and do { + if (system("cat '$target/$filename' | $hash_cmd > '$target/$filename.hash'")) { + die "Failed to generate hash for $filename\n"; + } + + my $sum = `cat "$target/$filename.hash"`; + $sum =~ /^(\w+)\s*/ or die "Could not generate file hash\n"; + $sum = $1; + + exit 0 if $sum eq $file_hash; + + die "Hash of the local file $filename does not match (file: $sum, requested: $file_hash) - deleting download.\n"; + unlink "$target/$filename"; + cleanup(); + }; +} + while (!-f "$target/$filename") { my $mirror = shift @mirrors; $mirror or die "No more mirrors to try - giving up.\n";