From patchwork Sat May 8 19:23:43 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sergey Ponomarev X-Patchwork-Id: 1475905 X-Patchwork-Delegate: ynezz@true.cz Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=none (no SPF record) smtp.mailfrom=lists.openwrt.org (client-ip=2001:8b0:10b:1:d65d:64ff:fe57:4e05; helo=desiato.infradead.org; envelope-from=openwrt-devel-bounces+incoming=patchwork.ozlabs.org@lists.openwrt.org; receiver=) Authentication-Results: ozlabs.org; dkim=pass (2048-bit key; secure) header.d=lists.infradead.org header.i=@lists.infradead.org header.a=rsa-sha256 header.s=desiato.20200630 header.b=T7p6Es/4; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.a=rsa-sha256 header.s=20161025 header.b=WG+7R+sp; dkim-atps=neutral Received: from desiato.infradead.org (desiato.infradead.org [IPv6:2001:8b0:10b:1:d65d:64ff:fe57:4e05]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 4Fcy3m6lWJz9tlZ for ; Sun, 9 May 2021 05:26:28 +1000 (AEST) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=desiato.20200630; h=Sender:Content-Transfer-Encoding :Content-Type:List-Subscribe:List-Help:List-Post:List-Archive: List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To:Message-Id:Date: Subject:Cc:To:From:Reply-To:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=zrr8VLMRnJrZABjr1Y9eHnU+NvKkKZ9c+aaoDoyaqZU=; b=T7p6Es/4D6EWlv16T2t4fPbAV HapU+InW7sQt+Ff1eUE1E4rTrs6TUnjImHktS2/krfn3ncpaD4Y5Ji7XVOn3osAD5ytiLssnA/+3C C5OZ2R+zGsi/QgHLrGmY8MklkdvSgfRBXLFuUSnl/bDh7U5OsD1PzDnMcsTLvvMpyt/YEoeTX1qcj g/vlSyEeZg4TklLZmXktPIVsns8O03v5bLquklza1VIgYGgQJg0hPR+Cme6ndhnLWK8wm5qNP1quv lGXEtbq2QJKZkY6tN7eLd3k/inSO57Tej1KWYE+aZrMN+2KNHpyUa3Hd8RsvIvLx+w6EkCBGTu48N OEUGslcpQ==; Received: from localhost ([::1] helo=desiato.infradead.org) by desiato.infradead.org with esmtp (Exim 4.94 #2 (Red Hat Linux)) id 1lfSYy-00AAid-Mr; Sat, 08 May 2021 19:24:32 +0000 Received: from mail-lj1-x236.google.com ([2a00:1450:4864:20::236]) by desiato.infradead.org with esmtps (Exim 4.94 #2 (Red Hat Linux)) id 1lfSYs-00AAhj-T6 for openwrt-devel@lists.openwrt.org; Sat, 08 May 2021 19:24:29 +0000 Received: by mail-lj1-x236.google.com with SMTP id w15so15786242ljo.10 for ; Sat, 08 May 2021 12:24:21 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=OHgIXHngjrxcaX82eAyVfv19bWJ2fP6vzlQPOdiQzgQ=; b=WG+7R+spICRvMmaKYQ6ZgNzDy7W5JuJ4/1WCy7pQWn8KwnnijDGAkclaVTE0vTdvQZ WpMCTe4RZoUVinXXXr8HqQw+441zBsvB1hxLu1+Hu75vNAEDqsB2uBc9H+j1tJM2dNO3 nKU9bRS0kwYCoCy6JWD7qtSHhMkdDH0k/eAm3u1vH+Me/3v1PULJfVegz0GA64xGJNFB GQky4hc+iW+KIl94DlRXPlTHRfnbZk48Un/kenOF4lHsTFT8IXOOko7P+CVW+f3+TJ0x oWiR14UVinBIp3O5f097cuIM4ay1fDcIdzAUBM/YD+BAaYJ1u9zkz4E2nxPe2/atKAGC fjMg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=OHgIXHngjrxcaX82eAyVfv19bWJ2fP6vzlQPOdiQzgQ=; b=crQnZbGuv4fGWBEdNrYQ/PQd67WxJ8vIakF1HQc//zk4HdEvJe+CI419j86C1CqUH7 /s8K5zRsl2FDoinTU+DrU5/JPEiN3grdlR7KfEB0EuB+hjYNTP7bMIpomvn2HMy968Ax nmiuhZyVDHHe4UhfeMDM7OXMcmjJaSpKY4ZvHR5101l4vcTQUPBmOv1zx3khpqVOhj5+ uueIS6sq8Yfq8c9h4q+JnGoqhoG5+Fomk6cY0P+/p6bkgmhYHGbFTdiZfc5Ne/GKd0Qn bt+gFSB/mDyw4pduAiaX7BRnQe0IKjcRlx+XslYxmBC/TUy9nWl4hUcAF/Fm3tRckuvF e+qg== X-Gm-Message-State: AOAM530xiNqA3nqUn92UBEI8+QRO5UBGKApmFNDdvRySYmp5HlBVFQX0 +13J6zvI6tdjONYp3eoQ2pLz4GoWGAs01w== X-Google-Smtp-Source: ABdhPJxK7JSupssctZ+Kq2+PPt8qj9j/r3aexee3zTzNVXb8wKBRmXV+zAazzq2COsotL7lwPQelDA== X-Received: by 2002:a05:651c:205e:: with SMTP id t30mr6527260ljo.388.1620501860421; Sat, 08 May 2021 12:24:20 -0700 (PDT) Received: from localhost.localdomain ([46.98.212.127]) by smtp.gmail.com with ESMTPSA id t1sm1691249lfg.226.2021.05.08.12.24.19 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sat, 08 May 2021 12:24:19 -0700 (PDT) From: Sergey Ponomarev To: openwrt-devel@lists.openwrt.org Cc: Sergey Ponomarev Subject: [PATCH] --header option to pass additional raw HTTP headers Date: Sat, 8 May 2021 22:23:43 +0300 Message-Id: <20210508192343.66941-1-stokito@gmail.com> X-Mailer: git-send-email 2.27.0 In-Reply-To: References: MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20210508_202427_065296_3481BB54 X-CRM114-Status: GOOD ( 17.66 ) X-Spam-Score: -0.2 (/) X-Spam-Report: Spam detection software, running on the system "desiato.infradead.org", has NOT identified this incoming email as spam. The original message has been attached to this so you can view it or label similar future email. If you have any questions, see the administrator of that system for details. Content preview: Signed-off-by: Sergey Ponomarev --- tests/cram/test-san_uclient-fetch.t | 1 + tests/cram/test_uclient-fetch.t | 1 + uclient-fetch.c | 16 +++++++++++++++ uclient-http.c | 31 +++++++++++++++++++++++++ [...] Content analysis details: (-0.2 points, 5.0 required) pts rule name description ---- ---------------------- -------------------------------------------------- -0.0 RCVD_IN_DNSWL_NONE RBL: Sender listed at https://www.dnswl.org/, no trust [2a00:1450:4864:20:0:0:0:236 listed in] [list.dnswl.org] -0.0 SPF_PASS SPF: sender matches SPF record 0.0 SPF_HELO_NONE SPF: HELO does not publish an SPF Record 0.0 FREEMAIL_FROM Sender email is commonly abused enduser mail provider [stokito[at]gmail.com] -0.1 DKIM_VALID Message has at least one valid DKIM or DK signature -0.1 DKIM_VALID_AU Message has a valid DKIM or DK signature from author's domain -0.1 DKIM_VALID_EF Message has a valid DKIM or DK signature from envelope-from domain 0.1 DKIM_SIGNED Message has a DKIM or DK signature, not necessarily valid X-BeenThere: openwrt-devel@lists.openwrt.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: OpenWrt Development List List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "openwrt-devel" Errors-To: openwrt-devel-bounces+incoming=patchwork.ozlabs.org@lists.openwrt.org Signed-off-by: Sergey Ponomarev --- tests/cram/test-san_uclient-fetch.t | 1 + tests/cram/test_uclient-fetch.t | 1 + uclient-fetch.c | 16 +++++++++++++++ uclient-http.c | 31 +++++++++++++++++++++++++++++ uclient.h | 1 + 5 files changed, 50 insertions(+) diff --git a/tests/cram/test-san_uclient-fetch.t b/tests/cram/test-san_uclient-fetch.t index 3158bde..3ed3be5 100644 --- a/tests/cram/test-san_uclient-fetch.t +++ b/tests/cram/test-san_uclient-fetch.t @@ -15,6 +15,7 @@ check uclient-fetch usage: \t--user=\t\t\tHTTP authentication username (esc) \t--password=\t\tHTTP authentication password (esc) \t--user-agent | -U \t\tSet HTTP user agent (esc) + \t--header='Header: value'\t\tAdd HTTP header. Multiple allowed (esc) \t--post-data=STRING\t\tuse the POST method; send STRING as the data (esc) \t--post-file=FILE\t\tuse the POST method; send FILE as the data (esc) \t--spider | -s\t\t\tSpider mode - only check file existence (esc) diff --git a/tests/cram/test_uclient-fetch.t b/tests/cram/test_uclient-fetch.t index 4ffe719..5a7ff43 100644 --- a/tests/cram/test_uclient-fetch.t +++ b/tests/cram/test_uclient-fetch.t @@ -15,6 +15,7 @@ check uclient-fetch usage: \t--user=\t\t\tHTTP authentication username (esc) \t--password=\t\tHTTP authentication password (esc) \t--user-agent | -U \t\tSet HTTP user agent (esc) + \t--header='Header: value'\t\tAdd HTTP header. Multiple allowed (esc) \t--post-data=STRING\t\tuse the POST method; send STRING as the data (esc) \t--post-file=FILE\t\tuse the POST method; send FILE as the data (esc) \t--spider | -s\t\t\tSpider mode - only check file existence (esc) diff --git a/uclient-fetch.c b/uclient-fetch.c index 282092e..4efc917 100644 --- a/uclient-fetch.c +++ b/uclient-fetch.c @@ -44,6 +44,7 @@ static const char *user_agent = "uclient-fetch"; static const char *post_data; static const char *post_file; +static const char **raw_headers = NULL; static struct ustream_ssl_ctx *ssl_ctx; static const struct ustream_ssl_ops *ssl_ops; static int quiet = false; @@ -342,6 +343,7 @@ static int init_request(struct uclient *cl) uclient_http_reset_headers(cl); uclient_http_set_header(cl, "User-Agent", user_agent); + uclient_http_set_raw_headers(cl, raw_headers); if (cur_resume) check_resume_offset(cl); @@ -481,6 +483,7 @@ static int usage(const char *progname) " --continue | -c Continue a partially-downloaded file\n" " --user= HTTP authentication username\n" " --password= HTTP authentication password\n" + " --header='Header: value' Add HTTP header. Multiple allowed\n" " --user-agent | -U Set HTTP user agent\n" " --post-data=STRING use the POST method; send STRING as the data\n" " --post-file=FILE use the POST method; send FILE as the data\n" @@ -542,6 +545,7 @@ enum { L_USER, L_PASSWORD, L_USER_AGENT, + L_HEADER, L_POST_DATA, L_POST_FILE, L_SPIDER, @@ -559,6 +563,7 @@ static const struct option longopts[] = { [L_USER] = { "user", required_argument, NULL, 0 }, [L_PASSWORD] = { "password", required_argument, NULL, 0 }, [L_USER_AGENT] = { "user-agent", required_argument, NULL, 0 }, + [L_HEADER] = { "header", required_argument, NULL, 0 }, [L_POST_DATA] = { "post-data", required_argument, NULL, 0 }, [L_POST_FILE] = { "post-file", required_argument, NULL, 0 }, [L_SPIDER] = { "spider", no_argument, NULL, 0 }, @@ -578,6 +583,7 @@ int main(int argc, char **argv) const char *proxy_url; char *username = NULL; char *password = NULL; + int raw_headers_count = 0; struct uclient *cl; int longopt_idx = 0; bool has_cert = false; @@ -626,6 +632,16 @@ int main(int argc, char **argv) case L_USER_AGENT: user_agent = optarg; break; + case L_HEADER: + if (!raw_headers) { + /* Max possible count of headers is the count of args (argc) - 2 + Since the first arg is program and last is a URL. + But user may forget the URL and raw_headers is null terminated so allocate argc */ + raw_headers = calloc(argc, sizeof(char *)); + } + raw_headers[raw_headers_count] = optarg; + raw_headers_count++; + break; case L_POST_DATA: post_data = optarg; break; diff --git a/uclient-http.c b/uclient-http.c index 349e69c..4ea40c7 100644 --- a/uclient-http.c +++ b/uclient-http.c @@ -96,6 +96,7 @@ struct uclient_http { uint32_t nc; + const char **raw_headers; struct blob_buf headers; struct blob_buf meta; }; @@ -587,6 +588,20 @@ uclient_http_add_auth_header(struct uclient_http *uh) return 0; } +static void +uclient_http_send_raw_headers(const struct uclient_http *uh) { + if (!uh->raw_headers) { + return; + } + const char **raw_headers = uh->raw_headers; + const char *raw_header = *raw_headers; + while (raw_header != NULL) { + ustream_printf(uh->us, "%s\r\n", raw_header); + raw_headers++; + raw_header = *raw_headers; + } +} + static int uclient_http_send_headers(struct uclient_http *uh) { @@ -625,6 +640,7 @@ uclient_http_send_headers(struct uclient_http *uh) if (err) return err; + uclient_http_send_raw_headers(uh); ustream_printf(uh->us, "\r\n"); uh->state = HTTP_STATE_HEADERS_SENT; @@ -1025,6 +1041,21 @@ uclient_http_set_header(struct uclient *cl, const char *name, const char *value) return 0; } +int +uclient_http_set_raw_headers(struct uclient *cl, const char **raw_headers) +{ + struct uclient_http *uh = container_of(cl, struct uclient_http, uc); + + if (cl->backend != &uclient_backend_http) + return -1; + + if (uh->state > HTTP_STATE_INIT) + return -1; + + uh->raw_headers = raw_headers; + return 0; +} + static int uclient_http_send_data(struct uclient *cl, const char *buf, unsigned int len) { diff --git a/uclient.h b/uclient.h index 4f37364..f1977bc 100644 --- a/uclient.h +++ b/uclient.h @@ -121,6 +121,7 @@ extern const struct uclient_backend uclient_backend_http; int uclient_http_reset_headers(struct uclient *cl); int uclient_http_set_header(struct uclient *cl, const char *name, const char *value); +int uclient_http_set_raw_headers(struct uclient *cl, const char **raw_headers); int uclient_http_set_request_type(struct uclient *cl, const char *type); int uclient_http_redirect(struct uclient *cl);