From patchwork Fri Jul 3 01:42:37 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sungbo Eo X-Patchwork-Id: 1322019 X-Patchwork-Delegate: mans0n@gorani.run Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=lists.openwrt.org (client-ip=2001:8b0:10b:1231::1; helo=merlin.infradead.org; envelope-from=openwrt-devel-bounces+incoming=patchwork.ozlabs.org@lists.openwrt.org; receiver=) Authentication-Results: ozlabs.org; dmarc=fail (p=none dis=none) header.from=gorani.run Authentication-Results: ozlabs.org; dkim=pass (2048-bit key; secure) header.d=lists.infradead.org header.i=@lists.infradead.org header.a=rsa-sha256 header.s=merlin.20170209 header.b=nFUvm188; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=gorani.run header.i=@gorani.run header.a=rsa-sha256 header.s=MBO0001 header.b=cOGix1nI; dkim-atps=neutral Received: from merlin.infradead.org (merlin.infradead.org [IPv6:2001:8b0:10b:1231::1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 49yd9T3m8qz9sRW for ; Fri, 3 Jul 2020 11:46:36 +1000 (AEST) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=merlin.20170209; h=Sender:Content-Transfer-Encoding: Content-Type:Cc:List-Subscribe:List-Help:List-Post:List-Archive: List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To:Message-Id:Date: Subject:To:From:Reply-To:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=gzi7pQJ6TR2S11EnNE6tqBHt4V+6ApFvdhPycHDBOZc=; b=nFUvm188T6PZIqnZsHdhJ8l+n OpVZFyatEPj7zwv1vRRJxBNmiQXASCHeTTNvrQYoSxOrEjA77LDrHDiH1LHpbkWLdc09UNbLtE+0e pYcW7t4AK6tiLBatUWMXdlE3S5TvKxm4uvG0vV72T8fmwBhdKWpsXSIRY/HA1hRvVxXluw6LzrAcO GWAvn7pOzPpa1eXf920dPqqeTMkCRf2sSTfCQx1+VycAP+rSgTVccnzcvrqxKiA8Idgdywgf29RG3 jyGtAczJckNTbataC9x7mO6lmP2M88bcuWoFlY48W6Jj0lzOM1qvn2OH8ciDIUp9EWpKNVHaCVv9o 5jVdtFeFw==; Received: from localhost ([::1] helo=merlin.infradead.org) by merlin.infradead.org with esmtp (Exim 4.92.3 #3 (Red Hat Linux)) id 1jrAjI-0005K2-7B; Fri, 03 Jul 2020 01:43:04 +0000 Received: from mout-p-202.mailbox.org ([80.241.56.172]) by merlin.infradead.org with esmtps (Exim 4.92.3 #3 (Red Hat Linux)) id 1jrAjF-0005IJ-CD for openwrt-devel@lists.openwrt.org; Fri, 03 Jul 2020 01:43:02 +0000 Received: from smtp1.mailbox.org (smtp1.mailbox.org [IPv6:2001:67c:2050:105:465:1:1:0]) (using TLSv1.2 with cipher ECDHE-RSA-CHACHA20-POLY1305 (256/256 bits)) (No client certificate requested) by mout-p-202.mailbox.org (Postfix) with ESMTPS id 49yd5B5p17zQlGD; Fri, 3 Jul 2020 03:42:54 +0200 (CEST) X-Virus-Scanned: amavisd-new at heinlein-support.de DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gorani.run; s=MBO0001; t=1593740573; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=4XkMPOsPOdzoIeq227xmJMZS9ywxr7BhEO1ZEhZsH7M=; b=cOGix1nIrj3LcgdLMZ/80ePnRgNBeWQttfN3ZNL7iHtgF7W/D2HP3LFY9B3v6q5SdR2GDL cfNZFYH+n6ILE8qswUBehuz14Wvc6fiH6lSUNae8AMOBhnrCt4/pkQ7h260Z8qDPMhTxYp OpLMqvrsl3eSjQaflZB6DU6XYd5ocIL1xYST+lCpUAN2saUdPG4MIQU10npLAi+W81KUXK H4tSOvFmGi2cAiq0of7y2oQLNJLA6mh2q9vhZInjJbNDbcJ2zcqSLEarDYbyP5gmOAb/Gg F8cJ2fgO43FmGhJbPY7GyyGAqUBlO0RtHBrGN67cRVrRufqCn/g+UpWZMGqcSg== Received: from smtp1.mailbox.org ([80.241.60.240]) by spamfilter03.heinlein-hosting.de (spamfilter03.heinlein-hosting.de [80.241.56.117]) (amavisd-new, port 10030) with ESMTP id QbZNa6ElS5JC; Fri, 3 Jul 2020 03:42:51 +0200 (CEST) From: Sungbo Eo To: openwrt-devel@lists.openwrt.org Subject: [PATCH uclient 1/2] uclient-fetch: fetch only header in spider mode Date: Fri, 3 Jul 2020 10:42:37 +0900 Message-Id: <20200703014238.19281-1-mans0n@gorani.run> In-Reply-To: <20200106134016.11354-1-mans0n@gorani.run> References: <20200106134016.11354-1-mans0n@gorani.run> MIME-Version: 1.0 X-MBO-SPAM-Probability: 0 X-Rspamd-Score: -5.79 / 15.00 / 15.00 X-Rspamd-Queue-Id: E17541800 X-Rspamd-UID: 11154f X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20200702_214301_623182_C89F3489 X-CRM114-Status: GOOD ( 11.44 ) X-Spam-Score: -0.9 (/) X-Spam-Report: SpamAssassin version 3.4.4 on merlin.infradead.org summary: Content analysis details: (-0.9 points) pts rule name description ---- ---------------------- -------------------------------------------------- -0.7 RCVD_IN_DNSWL_LOW RBL: Sender listed at https://www.dnswl.org/, low trust [80.241.56.172 listed in list.dnswl.org] 0.0 RCVD_IN_MSPIKE_H4 RBL: Very Good reputation (+4) [80.241.56.172 listed in wl.mailspike.net] -0.0 SPF_PASS SPF: sender matches SPF record 0.0 SPF_HELO_NONE SPF: HELO does not publish an SPF Record -0.1 DKIM_VALID Message has at least one valid DKIM or DK signature 0.1 DKIM_SIGNED Message has a DKIM or DK signature, not necessarily valid -0.1 DKIM_VALID_AU Message has a valid DKIM or DK signature from author's domain -0.1 DKIM_VALID_EF Message has a valid DKIM or DK signature from envelope-from domain 0.0 RCVD_IN_MSPIKE_WL Mailspike good senders X-BeenThere: openwrt-devel@lists.openwrt.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: OpenWrt Development List List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Sungbo Eo Sender: "openwrt-devel" Errors-To: openwrt-devel-bounces+incoming=patchwork.ozlabs.org@lists.openwrt.org Since spider mode only checks if a file exists, we don't need to download the whole file. This also matches wget's behavior. Signed-off-by: Sungbo Eo --- v2: rebased --- uclient-fetch.c | 16 +++++++++------- 1 file changed, 9 insertions(+), 7 deletions(-) diff --git a/uclient-fetch.c b/uclient-fetch.c index 061f0fd..95334b0 100644 --- a/uclient-fetch.c +++ b/uclient-fetch.c @@ -235,8 +235,12 @@ static void header_done_cb(struct uclient *cl) } case 204: case 200: - if (no_output) + if (no_output) { + if (!quiet) + fprintf(stderr, "Request succeeded\n"); + request_done(cl); break; + } if (tb[H_LEN]) out_len = strtoul(blobmsg_get_string(tb[H_LEN]), NULL, 10); @@ -265,7 +269,7 @@ static void read_data_cb(struct uclient *cl) ssize_t n; int len; - if (!no_output && output_fd < 0) + if (output_fd < 0) return; while (1) { @@ -274,11 +278,9 @@ static void read_data_cb(struct uclient *cl) return; out_bytes += len; - if (!no_output) { - n = write(output_fd, buf, len); - if (n < 0) - return; - } + n = write(output_fd, buf, len); + if (n < 0) + return; } } From patchwork Fri Jul 3 01:42:38 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sungbo Eo X-Patchwork-Id: 1322020 X-Patchwork-Delegate: mans0n@gorani.run Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=lists.openwrt.org (client-ip=2001:8b0:10b:1231::1; helo=merlin.infradead.org; envelope-from=openwrt-devel-bounces+incoming=patchwork.ozlabs.org@lists.openwrt.org; receiver=) Authentication-Results: ozlabs.org; dmarc=fail (p=none dis=none) header.from=gorani.run Authentication-Results: ozlabs.org; dkim=pass (2048-bit key; secure) header.d=lists.infradead.org header.i=@lists.infradead.org header.a=rsa-sha256 header.s=merlin.20170209 header.b=GjRKrqvx; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=gorani.run header.i=@gorani.run header.a=rsa-sha256 header.s=MBO0001 header.b=wo94rdsr; dkim-atps=neutral Received: from merlin.infradead.org (merlin.infradead.org [IPv6:2001:8b0:10b:1231::1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 49yd9T3KGHz9sQt for ; Fri, 3 Jul 2020 11:46:36 +1000 (AEST) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=merlin.20170209; h=Sender:Content-Transfer-Encoding: Content-Type:Cc:List-Subscribe:List-Help:List-Post:List-Archive: List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To:Message-Id:Date: Subject:To:From:Reply-To:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=vf2ZA4w3qgqMaDjVRZvrN2JBw3YaXUvJP/MqaztsQLo=; b=GjRKrqvxyccjBV87TUNrZ4gxf Igh56iOwoTrHi0KB/2xe5fyhZ48j7sETRMhut2NoHsajUUzpfc1z0VaqqscwsAIcLzwRYg/jaQqQC 5IiFVNw4d+TTRmLIOgMFV2zpfDupVbWwy9afIShrj33iZeOAvgVhjY0Lz1dfhFpskb1UEisJYwThg C09rzGVyz6LdszXroacLqYe2+JGeLIabSW9TFZ+bW3B4ZcYwTZmxdPosglrTm4YheGnvQuheOBklL DeZjtG+lRgh2Oll5tzcImWxLO8aKIq+RJ/xl6rIrlgLZeo7AspIUlUqWIgmYuzz4IHlbMuPILSx+h uZBf3Q9JQ==; Received: from localhost ([::1] helo=merlin.infradead.org) by merlin.infradead.org with esmtp (Exim 4.92.3 #3 (Red Hat Linux)) id 1jrAjJ-0005KF-KC; Fri, 03 Jul 2020 01:43:05 +0000 Received: from mout-p-102.mailbox.org ([2001:67c:2050::465:102]) by merlin.infradead.org with esmtps (Exim 4.92.3 #3 (Red Hat Linux)) id 1jrAjG-0005JG-S7 for openwrt-devel@lists.openwrt.org; Fri, 03 Jul 2020 01:43:03 +0000 Received: from smtp1.mailbox.org (smtp1.mailbox.org [80.241.60.240]) (using TLSv1.2 with cipher ECDHE-RSA-CHACHA20-POLY1305 (256/256 bits)) (No client certificate requested) by mout-p-102.mailbox.org (Postfix) with ESMTPS id 49yd5J4fcPzKmDL; Fri, 3 Jul 2020 03:43:00 +0200 (CEST) X-Virus-Scanned: amavisd-new at heinlein-support.de DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gorani.run; s=MBO0001; t=1593740578; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=Cr5GoufFvPE7YKxsXxUhTY4qOC/1WtOxhS6/qroZ988=; b=wo94rdsrBb4+bszyQqyUWy2WG9lFD1zXRbwRRzS0Qk2BQ6wcNc86tQ0ZIC9K1UCO6FFvr7 rDR79nJsHyBMVQqLyvDc69ZgEAoxcvv3/2piQBkH1zxwqZrDrQTvFAcFxfwd29wR+xlY0D H4VF8ljNDoT6KE0DTc2NIJld3UogJQFKROVrH6jY5FB54EUlclsE4mt3bolAmw5c2vi5Z8 rmyhjjDJkjppiRWIdAUMgaGnXOl0sryrjy/E3ijcFYGHHW5U2sWx53i3L1rDpGM8b9JvjB Q7UDp5EiT056h0iKdneEgEcL+ZNaafcE6Ma+2WI04B09vgTjdGGUfswPhVFcuw== Received: from smtp1.mailbox.org ([80.241.60.240]) by spamfilter01.heinlein-hosting.de (spamfilter01.heinlein-hosting.de [80.241.56.115]) (amavisd-new, port 10030) with ESMTP id fU6qfk-A28iO; Fri, 3 Jul 2020 03:42:57 +0200 (CEST) From: Sungbo Eo To: openwrt-devel@lists.openwrt.org Subject: [PATCH uclient 2/2] uclient-fetch: use HEAD method in spider mode Date: Fri, 3 Jul 2020 10:42:38 +0900 Message-Id: <20200703014238.19281-2-mans0n@gorani.run> In-Reply-To: <20200703014238.19281-1-mans0n@gorani.run> References: <20200106134016.11354-1-mans0n@gorani.run> <20200703014238.19281-1-mans0n@gorani.run> MIME-Version: 1.0 X-MBO-SPAM-Probability: 0 X-Rspamd-Score: -5.79 / 15.00 / 15.00 X-Rspamd-Queue-Id: B78AE1758 X-Rspamd-UID: ecbd22 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20200702_214303_027607_90F5402A X-CRM114-Status: GOOD ( 13.19 ) X-Spam-Score: -0.2 (/) X-Spam-Report: SpamAssassin version 3.4.4 on merlin.infradead.org summary: Content analysis details: (-0.2 points) pts rule name description ---- ---------------------- -------------------------------------------------- -0.0 SPF_PASS SPF: sender matches SPF record 0.0 SPF_HELO_NONE SPF: HELO does not publish an SPF Record -0.1 DKIM_VALID Message has at least one valid DKIM or DK signature 0.1 DKIM_SIGNED Message has a DKIM or DK signature, not necessarily valid -0.1 DKIM_VALID_AU Message has a valid DKIM or DK signature from author's domain -0.1 DKIM_VALID_EF Message has a valid DKIM or DK signature from envelope-from domain X-BeenThere: openwrt-devel@lists.openwrt.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: OpenWrt Development List List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Sungbo Eo Sender: "openwrt-devel" Errors-To: openwrt-devel-bounces+incoming=patchwork.ozlabs.org@lists.openwrt.org HEAD method does not request response body, so it is suitable for spider mode. This also matches wget's behavior. Signed-off-by: Sungbo Eo --- v2: rebased --- uclient-fetch.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/uclient-fetch.c b/uclient-fetch.c index 95334b0..2d201ef 100644 --- a/uclient-fetch.c +++ b/uclient-fetch.c @@ -337,7 +337,7 @@ static int init_request(struct uclient *cl) msg_connecting(cl); - rc = uclient_http_set_request_type(cl, post_data || post_file ? "POST" : "GET"); + rc = uclient_http_set_request_type(cl, post_data || post_file ? "POST" : no_output ? "HEAD" : "GET"); if (rc) return rc;