From patchwork Mon Nov 20 15:54:55 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Stefano Babic X-Patchwork-Id: 1866123 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=googlegroups.com header.i=@googlegroups.com header.a=rsa-sha256 header.s=20230601 header.b=aUROld8E; dkim-atps=neutral Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=googlegroups.com (client-ip=2a00:1450:4864:20::13c; helo=mail-lf1-x13c.google.com; envelope-from=swupdate+bncbaabbwmc52vamgqexodzwrq@googlegroups.com; receiver=patchwork.ozlabs.org) Received: from mail-lf1-x13c.google.com (mail-lf1-x13c.google.com [IPv6:2a00:1450:4864:20::13c]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (secp384r1) server-digest SHA384) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4SYsXY61lfz1ySY for ; Tue, 21 Nov 2023 02:55:09 +1100 (AEDT) Received: by mail-lf1-x13c.google.com with SMTP id 2adb3069b0e04-50aa95b5f99sf1993186e87.2 for ; Mon, 20 Nov 2023 07:55:09 -0800 (PST) ARC-Seal: i=2; a=rsa-sha256; t=1700495706; cv=pass; d=google.com; s=arc-20160816; b=tKJkWGoQqXmWaSvLOGXvh+D3Rj73qfODl37Yn3r+vT/viyBrZwJUA5ewpEQDPQNtni GNkC5NDH6EZWxSOs8j0pwHnst4IsKVeen6QdYHMF7O3H8XdXvuJ5qCuhD2l5KdqV2RG4 iOSoXDOl+baxeo5cXegZHX06yTks3Aan0wFqFSz64dhyc/RXj9qw3VXS4x5/ywp18kFu 7N3t5WlUy56xgKwXZyIIfA4RnU3tI7qz9WXnl6PbLzmnggj1Lb7ry/ijHrv4YNXDY/WE tn8okUVKZGjnXCUzCcP1G7NVOeTSMgL31nDkfQ/cuL6ZPW9Y3ENhIFUevkvSV/GQ3obn YQCA== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-unsubscribe:list-subscribe:list-archive:list-help:list-post :list-id:mailing-list:precedence:ui-outboundreport:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from:sender :dkim-signature; bh=/Js1CJ/DxlVFwCdT+9bfGKx4ONFJ7k/+leuEBeEIjxw=; fh=TiEWcqMcKpHk5s7uErzpntoONrNfOXwKpI5P8bIlggk=; b=j/a4sU/09E5c1SRJNWvtvDsjwXUPWBwir2dI/nJY0QlGYt/JrwA73H3qv4ncuXJ6Bt xu0DHpnoCuMRuFRykMse8wrwpJocBrsgc7HL/3jZWc0fe7puUn/Qq+X/ktDuS7f5KclU SbblR3izeSad0vdrltWqfxf8gMIYydHfDfz3CwqubiRu6ojJclhGLgZHOQ3Q0+47S74d vSbi9ZckwkeoXK/+Hbzle5hhW4iCCI0Yxs4JjzworOBGRQ88YkcvBYWwd4JvCPeuZ6Io 3FLCTwQm0SL3dQ9BZO0VrWKocJTQa/cVnXZQx/iIsyTCxTS+Dv4vd8aMQ98pH77DerqN 7O6g== ARC-Authentication-Results: i=2; gmr-mx.google.com; spf=pass (google.com: domain of stefano.babic@swupdate.org designates 212.227.17.24 as permitted sender) smtp.mailfrom=stefano.babic@swupdate.org DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=googlegroups.com; s=20230601; t=1700495706; x=1701100506; darn=patchwork.ozlabs.org; h=list-unsubscribe:list-subscribe:list-archive:list-help:list-post :list-id:mailing-list:precedence:x-original-authentication-results :x-original-sender:ui-outboundreport:mime-version:references :in-reply-to:message-id:date:subject:cc:to:from:sender:from:to:cc :subject:date:message-id:reply-to; bh=/Js1CJ/DxlVFwCdT+9bfGKx4ONFJ7k/+leuEBeEIjxw=; b=aUROld8E14mbQtgVYC+4MdYpUygyYUO2UPXhsysjT1ReFBxM0NbYhPewI263vexyS8 lJ8ScFQUFvJHW8pQL2hEUlN3+Pywl2t1AmmEUzgcEH0wD35Mhf+9l2CJLZoYn0fHlm2Q CxAmIatdR+E49zAVKAUGf88rG3+CpwxTnpOJ7oXpIG2BUCqu+DI5AzT3f7Jb7GsWBFFG g8cZBSITV9mECc3gk2CxUI9sbVxW6CFcSX4ljAP18ShDlDO+0ShDJ+1fJ2eDVbsbF/0S G9PKdWD75rtQShTt7NFOK2y1C+KnK60A44iQL+0PTX96E/hwikeatEOoxJNpqah9v9Oo s+og== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1700495706; x=1701100506; h=list-unsubscribe:list-subscribe:list-archive:list-help:list-post :x-spam-checked-in-group:list-id:mailing-list:precedence :x-original-authentication-results:x-original-sender :ui-outboundreport:mime-version:references:in-reply-to:message-id :date:subject:cc:to:from:x-beenthere:x-gm-message-state:sender:from :to:cc:subject:date:message-id:reply-to; bh=/Js1CJ/DxlVFwCdT+9bfGKx4ONFJ7k/+leuEBeEIjxw=; b=Htz8g//taIVIgCQRt2ekNSb29+6dLTNvv/1iT7TG2GFO6L5/2svgyGQUzzA1Qz+qr2 kYuYA9cB6OVJxEUVr3oR1U5/Bl0ErND8wEyZRAMmrTj96DTdWh7HulkXStGncLG0+/gU BFquPcGqmorhrpURMtdKCaaZRy/JATTYGpuVlSTI/w+xGUu3r7+w8Kn5exdw5yZO3eqR tR60LTfUYMTekkvw0x2p6xjE9jvTf7b88qphpFdEqjrQLKLw8O7odaAediuaGai3BzYu elaHCgXd1Ec1rhgT3gyE8His2gtvTMXvp7NDY8shSszvsQSIzJTO4DIORpzlYTKbgCHq 0L0A== Sender: swupdate@googlegroups.com X-Gm-Message-State: AOJu0YzY09LU33I7pZ81/Fc6mv5Bhu2e/z6N/KYAob2kaoCjKjePi4uL MCIB+q430clcCQ8w2mzA/Pc= X-Google-Smtp-Source: AGHT+IGKniiAJBd9+HeHI7a6mVBaJErxMyNzk1JUzweG9KfokbNTiOA13ZQXH62aB8LCOELqwdPpLA== X-Received: by 2002:ac2:4423:0:b0:502:9fce:b6da with SMTP id w3-20020ac24423000000b005029fceb6damr4995964lfl.21.1700495705939; Mon, 20 Nov 2023 07:55:05 -0800 (PST) X-BeenThere: swupdate@googlegroups.com Received: by 2002:a05:6512:b16:b0:4f9:5599:26a with SMTP id w22-20020a0565120b1600b004f95599026als996299lfu.2.-pod-prod-08-eu; Mon, 20 Nov 2023 07:55:04 -0800 (PST) X-Received: by 2002:a05:6512:41e:b0:509:31e6:1de5 with SMTP id u30-20020a056512041e00b0050931e61de5mr5171059lfk.47.1700495703933; Mon, 20 Nov 2023 07:55:03 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1700495703; cv=none; d=google.com; s=arc-20160816; b=iUdFCpNTkYAkT2a3peI9lcM5jmHQXPJXRk6kfN+M4Vrso9D4RxN/X/TpCPSx7Y1Vne Tq4M/twGulsiC8Mye3zgJqIXflq3eETxWDzkBhxmeS7Jz/C74BNAWIKcIPmAW6Wq4vci a7/P+Ug/fC98ymIwLkMWZAW9xoqZr4hyD3d6sGDH7jBmMQCkh+DNLUUzYQF8hRry6owr CWfGe8PX5wUkelfLvBWCIkwHOcIaEkcunh143jazzn4vAFEX2RwvRaY3Nc8Ico+YVcvn gW6rUY5WN9gQmGR8FydBUilms/nn2lIcxTdJqY0aMSMSSHM8FT0I3tHDF3Siq7NRcpVS T+6w== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=ui-outboundreport:content-transfer-encoding:mime-version:references :in-reply-to:message-id:date:subject:cc:to:from; bh=HFuI6g6Vi3xp013LBIDZ7viy7AXQiOwTRCSjNFSWWiM=; fh=TiEWcqMcKpHk5s7uErzpntoONrNfOXwKpI5P8bIlggk=; b=pO4q0y9C5giMnFtWyZWj8MUqz/srGRNuYTP/auA45uAnKhL5x1ogr4wzK+jouxsehz r/YogIkf5PWBVtE0c2/uCRve2VY1wGHOUwy2uoO8cZSHwGjGtxdb2tMJniJ6Ky3xGloQ hnFXON6gCJrEZp8gz2UZFQUJPg5/59wOksB5O8dxiH93sOWRzoGlkF/O4Z+uszYOkDds nEaMzWT+Zr2VmqAoXKx4VvwUzJs2Yk5/cDnBPgtoXwcm2sRwh/9/cUIqlFvY70oWc+Oa KFyuo4r1CmRl+o0/2uShCY3r7KfhXLbMC1S22gq3yW5hXPD4J7S4laIWJH06BnyaRmSU LEwA== ARC-Authentication-Results: i=1; gmr-mx.google.com; spf=pass (google.com: domain of stefano.babic@swupdate.org designates 212.227.17.24 as permitted sender) smtp.mailfrom=stefano.babic@swupdate.org Received: from mout.kundenserver.de (mout.kundenserver.de. [212.227.17.24]) by gmr-mx.google.com with ESMTPS id f7-20020a05651232c700b005008765a16fsi297489lfg.13.2023.11.20.07.55.03 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 20 Nov 2023 07:55:03 -0800 (PST) Received-SPF: pass (google.com: domain of stefano.babic@swupdate.org designates 212.227.17.24 as permitted sender) client-ip=212.227.17.24; Received: from paperino.fritz.box ([88.217.136.221]) by mrelayeu.kundenserver.de (mreue109 [213.165.67.113]) with ESMTPSA (Nemesis) id 1MGQax-1r8C5K10wv-00GnLB; Mon, 20 Nov 2023 16:55:03 +0100 From: Stefano Babic To: swupdate@googlegroups.com Cc: Stefano Babic Subject: [swupdate] [PATCH 09/13] channel_curl: implement put_file entry point Date: Mon, 20 Nov 2023 16:54:55 +0100 Message-Id: <20231120155459.45188-10-stefano.babic@swupdate.org> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20231120155459.45188-1-stefano.babic@swupdate.org> References: <20231120155459.45188-1-stefano.babic@swupdate.org> MIME-Version: 1.0 X-Provags-ID: V03:K1:iZ79sa1ueT2vrHkJ/Hp1i4bG3AkYDqSP8bQSw8WYHZz4L1FMEko 9xoQYNBdrywLnQushFyQTE+/pz5jQnOQORYJp24DQSzNITGMELu+ABitUjFViN4VyuIOO+g KM1XHJQqauuqvIv1xt1vnLquzRRN3ektV58rVN8tH28U6UwHl/4s60Gm8pJLUfpjNC/mSmN McCcQe/JyKu+UparMndjA== X-Spam-Flag: NO UI-OutboundReport: notjunk:1;M01:P0:BfGelg/QMlY=;UuEbM0kzi1sSGjmj/z3h2CWzm0s fioFZcUErLTxaJn9IyIyqOoFMnmuvNll9Nb80QJtirbJxRXi6kEMWH0rtzH8nK/x0PexNdLXY RjfR6U/qr8eGp1Y/MomtflGpj8yms6094+EkjZxXwYMGC9yWWKePyEL6sFiLxDgtaohFPJ4KB zcLyWh7GvDQ8n/CobM+RCtusWxmBFFLXc7128R0RR1lISiDbC92+xxrdTBLGnX1QSB2LsT+vO 7eNgUYA72NsnYlDFq9LvNeR2dDyC8CYR8A1H+uMbVSPynMySFy/2j3F+ivUY7LI/kxG/2yxlD Z9FP2f75OHHc00hwcDrEkWhFvGLyf8dFA+Lt5b17Guip+DHHAfAx93F2TuId5mtd6QXb5nHcb jOEcUao8pgNqHqtQ5hH2jRFtH1AwsbUi836pnxusW09/87VFfJamSoRMrGoOGDPsHvfCzpJ9W TIEw2ds5SGPvHn4sFETo7lg1rqfo09+tOkNp/PXwad79jaZ/hPMwDzP10rWG+bZW/nXa9i/F3 zuFesfE+ywGY2shaVBfZ+VQSLolRSHoM7H5086e/ouDi3MvgmGDnOUpwLQ0/fFLwNvYQ1G/Mv x+Do8v1cLdLKEl6O2zIJTdsstDgor8nqaLPoLn9GWysKSiv+XYsFeKd/stqo4CxgZ/Jo9fa3k TZ7OAGmghPlRs1FVt/JnI80Osl9QltVv9RCR9bFybaBlUcKIk0E4xriTOg/EtGcVnv5jr3ou3 Qi08dX+0bImMTMgUnYH0PAG0SWSmsjUOFWBk3SP0kQuRIPxZpGr5H9ZlUOiBatH1OtsbyfgVW QosX9w7l8uRqWIUlNjtu3KkGZOnXr7Rs7aayGcF4TbYvtTYm5cLi7bjQ7powzeAqJH0KwN03V QX2Q1SS71pPf5UxUznOTKIS5mwuUDU859az0= X-Original-Sender: stefano.babic@swupdate.org X-Original-Authentication-Results: gmr-mx.google.com; spf=pass (google.com: domain of stefano.babic@swupdate.org designates 212.227.17.24 as permitted sender) smtp.mailfrom=stefano.babic@swupdate.org Precedence: list Mailing-list: list swupdate@googlegroups.com; contact swupdate+owners@googlegroups.com List-ID: X-Spam-Checked-In-Group: swupdate@googlegroups.com X-Google-Group-Id: 605343134186 List-Post: , List-Help: , List-Archive: , List-Unsubscribe: , Use put_file to upload a file from a stream. Data is read from a fd that must be set by the caller into channel_data_t structure. Caller must set the total amoun t of data that must be uploaded, else libcurl cannot know. Signed-off-by: Stefano Babic --- corelib/channel_curl.c | 125 +++++++++++++++++++++++++++++++++++------ include/channel_curl.h | 1 + 2 files changed, 110 insertions(+), 16 deletions(-) diff --git a/corelib/channel_curl.c b/corelib/channel_curl.c index 4a209bcd..27b31247 100644 --- a/corelib/channel_curl.c +++ b/corelib/channel_curl.c @@ -86,6 +86,7 @@ static channel_op_res_t channel_open(channel_t *this, void *cfg); static channel_op_res_t channel_get(channel_t *this, void *data); static channel_op_res_t channel_get_file(channel_t *this, void *data); static channel_op_res_t channel_put(channel_t *this, void *data); +static channel_op_res_t channel_put_file(channel_t *this, void *data); channel_op_res_t channel_curl_init(void); channel_t *channel_new(void); @@ -119,6 +120,7 @@ channel_t *channel_new(void) newchan->get = &channel_get; newchan->get_file = &channel_get_file; newchan->put = &channel_put; + newchan->put_file = &channel_put_file; newchan->get_redirect_url = &channel_get_redirect_url; } @@ -888,12 +890,11 @@ cleanup: return result; } -static size_t read_callback(void *ptr, size_t size, size_t nmemb, void *data) +static size_t read_callback(char *ptr, size_t size, size_t nmemb, void *data) { channel_data_t *channel_data = (channel_data_t *)data; - ssize_t bytes; + ssize_t nbytes; size_t n = 0; - int ret; /* * Check if data is stored in a buffer or should be read @@ -901,30 +902,34 @@ static size_t read_callback(void *ptr, size_t size, size_t nmemb, void *data) */ if (channel_data->request_body) { /* Check data to be sent */ - bytes = strlen(channel_data->request_body) - channel_data->offs; + nbytes = strlen(channel_data->request_body) - channel_data->offs; - if (!bytes) + if (!nbytes) return 0; - n = min(bytes, size * nmemb); + n = min(nbytes, size * nmemb); memcpy(ptr, &channel_data->request_body[channel_data->offs], n); channel_data->offs += n; } else { - bytes = nmemb * size; - ret = read(channel_data->read_fifo, ptr, bytes); - if (ret < 0) { + if (nmemb * size > channel_data->upload_filesize) + nbytes = channel_data->upload_filesize; + else + nbytes = nmemb * size; + + nbytes = read(channel_data->read_fifo, ptr, nbytes); + if (nbytes < 0) { if (errno == EAGAIN) { TRACE("READ EAGAIN"); - bytes = 0; + nbytes = 0; } else { ERROR("Cannot read from FIFO"); return CURL_READFUNC_ABORT; } - } else - bytes = ret; + } - n = bytes / nmemb; + n = nbytes / size; + channel_data->upload_filesize -= nbytes; } return n; @@ -1018,9 +1023,8 @@ static CURLcode channel_set_read_callback(channel_curl_t *handle, channel_data_t { return curl_easy_setopt(handle, CURLOPT_READFUNCTION, read_callback) || - (channel_data->request_body ? - curl_easy_setopt(handle, CURLOPT_INFILESIZE_LARGE, - (curl_off_t)strlen(channel_data->request_body)) : 0) || + curl_easy_setopt(handle, CURLOPT_INFILESIZE_LARGE, + channel_data->request_body ? (curl_off_t)strlen(channel_data->request_body) : (curl_off_t)channel_data->upload_filesize) || curl_easy_setopt(handle, CURLOPT_READDATA, channel_data); } @@ -1136,6 +1140,95 @@ channel_op_res_t channel_put(channel_t *this, void *data) } } +channel_op_res_t channel_put_file(channel_t *this, void *data) +{ + CURLcode curl_result = CURLE_OK; + channel_curl_t *channel_curl = this->priv; + assert(data != NULL); + assert(channel_curl->handle != NULL); + + channel_op_res_t result = CHANNEL_OK; + channel_data_t *channel_data = (channel_data_t *)data; + channel_data->offs = 0; + output_data_t outdata = {}; + write_callback_t wrdata = { .this = this, .channel_data = channel_data, .outdata = &outdata }; + + if ((result = channel_set_content_type(this, channel_data)) != + CHANNEL_OK) { + ERROR("Set content-type option failed."); + goto cleanup_header; + } + + if ((result = channel_set_options(this, channel_data)) != CHANNEL_OK) { + ERROR("Set channel option failed."); + goto cleanup_header; + } + + if ((result = setup_reply_buffer(channel_curl->handle, &wrdata)) != CHANNEL_OK) { + goto cleanup_header; + } + + if (!channel_data->method) + channel_data->method = CHANNEL_POST; + + switch (channel_data->method) { + case CHANNEL_PATCH: + case CHANNEL_POST: + if (channel_data->method == CHANNEL_PATCH) + curl_result |= curl_easy_setopt(channel_curl->handle, CURLOPT_CUSTOMREQUEST, "PATCH"); + else + curl_result |= curl_easy_setopt(channel_curl->handle, CURLOPT_POST, 1L); + + break; + + case CHANNEL_PUT: + curl_result |= curl_easy_setopt(channel_curl->handle, + #if LIBCURL_VERSION_NUM >= 0x70C01 + CURLOPT_UPLOAD, + #else + CURLOPT_PUT, + #endif + 1L); + break; + } + + curl_result |= channel_set_read_callback(channel_curl->handle, channel_data); + if (curl_result != CURLE_OK) { + result = CHANNEL_EINIT; + ERROR("Set %s channel method option failed.", method_desc[channel_data->method]); + goto cleanup_header; + } + + CURLcode curlrc = curl_easy_perform(channel_curl->handle); + if (curlrc != CURLE_OK) { + ERROR("Channel %s operation failed (%d): '%s'", method_desc[channel_data->method], curlrc, + curl_easy_strerror(curlrc)); + result = channel_map_curl_error(curlrc); + goto cleanup_header; + } + + channel_log_effective_url(this); + + result = channel_map_http_code(this, &channel_data->http_response_code); + + if (channel_data->nocheckanswer) + goto cleanup_header; + + channel_log_reply(result, channel_data, &outdata); + + if (result == CHANNEL_OK) { + result = parse_reply(channel_data, &outdata); + } + +cleanup_header: + outdata.memory != NULL ? free(outdata.memory) : (void)0; + curl_easy_reset(channel_curl->handle); + curl_slist_free_all(channel_curl->header); + channel_curl->header = NULL; + + return result; +} + channel_op_res_t channel_get_file(channel_t *this, void *data) { channel_curl_t *channel_curl = this->priv; diff --git a/include/channel_curl.h b/include/channel_curl.h index b1fc65c2..b3b51e85 100644 --- a/include/channel_curl.h +++ b/include/channel_curl.h @@ -90,6 +90,7 @@ typedef struct { struct dict *headers_to_send; struct dict *received_headers; unsigned int max_download_speed; + size_t upload_filesize; char *range; /* Range request for get_file in any */ void *user; } channel_data_t;