From patchwork Thu Jan 11 16:28:30 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: =?utf-8?q?Jakub_Hor=C3=A1k?= X-Patchwork-Id: 859196 X-Patchwork-Delegate: blogic@openwrt.org Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=none (mailfrom) smtp.mailfrom=lists.infradead.org (client-ip=65.50.211.133; helo=bombadil.infradead.org; envelope-from=lede-dev-bounces+incoming=patchwork.ozlabs.org@lists.infradead.org; receiver=) Authentication-Results: ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=lists.infradead.org header.i=@lists.infradead.org header.b="eSEDjUtV"; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=braiins-cz.20150623.gappssmtp.com header.i=@braiins-cz.20150623.gappssmtp.com header.b="HOtSs7hK"; dkim-atps=neutral Received: from bombadil.infradead.org (bombadil.infradead.org [65.50.211.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 3zHWXz6cklz9sNV for ; Fri, 12 Jan 2018 03:29:07 +1100 (AEDT) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20170209; h=Sender:Content-Type: List-Subscribe:List-Help:List-Post:List-Archive:List-Unsubscribe:List-Id: Subject:MIME-Version:Date:Message-ID:From:To:Reply-To:Cc: Content-Transfer-Encoding:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:In-Reply-To: References:List-Owner; bh=gWYaO7MojJZpdHYoCXRs726Bxp2H4Onx5ei5JGaruO0=; b=eSE DjUtVHFfmNMAmz6c+BhIpQBE8D2k9IApciajgYHvIrgHFEffsytDfU4LQZu4JdawLeGck5kJm/vIf nMw2wzgwfPSam2ZUDkt/AkyG/fElI5Na+jRH5hnUokeBig8lj43jqanCN9YO4Jmj3749l19cdJRoq 3uq0IPPZhVP7e3Y7p8s/NFYQozFncaXCdIPaGjWbGkEeo9+kVcO42CIZThn2vZrpNreCnEVi67DqC C8m4OcC8mChYkzHy83/1AmgbJks+JfhxADy4fX2+X5KpAxWRuuw4jEHuvlKZo29D5Xjveh0AF4y8G FwKuJz7SqJqE6bU9KHu0sC0EkDKBawQ==; Received: from localhost ([127.0.0.1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.89 #1 (Red Hat Linux)) id 1eZfin-0004oi-Lv; Thu, 11 Jan 2018 16:28:53 +0000 Received: from mail-wr0-x22c.google.com ([2a00:1450:400c:c0c::22c]) by bombadil.infradead.org with esmtps (Exim 4.89 #1 (Red Hat Linux)) id 1eZfik-0004me-6s for lede-dev@lists.infradead.org; Thu, 11 Jan 2018 16:28:52 +0000 Received: by mail-wr0-x22c.google.com with SMTP id w50so2702013wrc.11 for ; Thu, 11 Jan 2018 08:28:39 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=braiins-cz.20150623.gappssmtp.com; s=20150623; h=to:from:subject:message-id:date:user-agent:mime-version; bh=2F1Axv63EWf9eJsg+olUZQ4jrlvoxH2VvMRWtAPkxVI=; b=HOtSs7hKHXu38nT/3ZhoP8RuE2TmcgQ+/5DfDNFGt5fvl/V5dV4fU1FZUwZH4XrSAj d7V4Wg6zwoIOIGc20/wVnpHifjE9t5U1c2LspdAKKvIxsV1jnf2Mh0rGL99+km/CHcqU pnkX05FSQoBr4qcDb9NlgcpQsMW9J/x6PMzsfch1QKy5hAtpssS+78V7KyEfmkP9peTc tDENswyuz6Zjie0gkHL3q2OXbi6ABXLF6vHgb/Rk54Shzr+X3phAXdunM/Ivm+Wi6ljW No+96sKemPgRrN82SkmfO2Je7rxuFm4yTZGuyBXEZbrQ7Zw5IsIerLrlRMtVsqQYm+3f qfCA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:to:from:subject:message-id:date:user-agent :mime-version; bh=2F1Axv63EWf9eJsg+olUZQ4jrlvoxH2VvMRWtAPkxVI=; b=a6jRGNnq2qCGEMmQqr4qhOlRZfXlF58ARXG+6y8EzkUZXtAmLPK+uyt/Lw+pXmGN91 76kTEM15jT1rafcxn8ApWrSNs2NFTZyYJdQFwnOS785QnUkeCc9TucghCMKOWFbEHuDC yCpFP+KjPXmiW24hd69wo61bDBzVrEujfWv2/yQdMP3m/vqjMvVbkCNM3MlXjdFdjjEX OxkZuIZENhU03MZt4gEiY2mSqajYZMSyOI1aRH8L3OdV6zFWqxnPe8AjQnQcdkjCbfyI mv3osBnwxVyBTnDPOatmT4I6+sWiuEUTtRezuZYUytAmf1W9yd7UlhGKcNUeV8GLSBSl t/kA== X-Gm-Message-State: AKGB3mI/zl+/yD5PW35K3Cexs0P+kEI1meWhhUQdDKdEfr0HvKI5jjZQ ftY/gqwyWRRXxWPlPb7Zhczx+A== X-Google-Smtp-Source: ACJfBotz2lufWeT+W7kOQ0BBZQAetzgXPxbQCM+D9Xaway85QQl1hpQ1HV56JlmaehJwbmw9v9EDqQ== X-Received: by 10.223.178.26 with SMTP id u26mr19801339wra.149.1515688117853; Thu, 11 Jan 2018 08:28:37 -0800 (PST) Received: from [10.33.0.244] ([94.230.156.78]) by smtp.gmail.com with ESMTPSA id q6sm19983921wrc.36.2018.01.11.08.28.36 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Thu, 11 Jan 2018 08:28:36 -0800 (PST) To: lede-dev@lists.infradead.org From: =?utf-8?q?Jakub_Hor=C3=A1k?= Message-ID: <173cfe61-4966-2870-ce78-677385a2602a@braiins.cz> Date: Thu, 11 Jan 2018 17:28:30 +0100 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:52.0) Gecko/20100101 Thunderbird/52.4.0 MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20180111_082850_329883_039545C6 X-CRM114-Status: GOOD ( 20.10 ) X-Spam-Score: -1.9 (-) X-Spam-Report: SpamAssassin version 3.4.1 on bombadil.infradead.org summary: Content analysis details: (-1.9 points) pts rule name description ---- ---------------------- -------------------------------------------------- -0.0 RCVD_IN_DNSWL_NONE RBL: Sender listed at http://www.dnswl.org/, no trust [2a00:1450:400c:c0c:0:0:0:22c listed in] [list.dnswl.org] -1.9 BAYES_00 BODY: Bayes spam probability is 0 to 1% [score: 0.0000] -0.1 DKIM_VALID Message has at least one valid DKIM or DK signature 0.1 DKIM_SIGNED Message has a DKIM or DK signature, not necessarily valid Subject: [LEDE-DEV] Bug when processing long lines X-BeenThere: lede-dev@lists.infradead.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "Lede-dev" Errors-To: lede-dev-bounces+incoming=patchwork.ozlabs.org@lists.infradead.org Hello LEDE developers, I found a bug in procd that gets triggered when long lines are printed by services whose stdout/stderr are being logged. The bug itself is explained in the attached patch. However, when I was testing the fix, I found out that the bug is present in other places, mostly those that call "ustream_get_read_buf" function. Some examples: - ubox/log/logread.c:logread_fb_data_cb() - when buffer passed on the descriptor is larger than 4096, reception stops - netifd/main.c:netifd_process_log_read_cb - this is a place that seems to have this bug fixed, but still has incorrect handling of NUL bytes - libubox/examples/ustream-example.c:client_read_cb - this is probably the place that originated this broken bit of code - uhttpd/relay.c:relay_process_headers - another place that seems broken I've attached an init script (that goes to /etc/init.d/flood) and three Lua programs (flood[123].lua) that trigger this behavior: - flood1.lua writes long message to stdout, that triggers this behavior in procd - flood2.lua writes message that gets correctly processed by procd, but triggers the bug in logread - flood3.lua writes message with embedded zeros I don't post patches to mailing lists very often, so I apologize if I'm sending this in a wrong format or in a too broken english. Best regards, Jakub Horak #!/bin/sh /etc/rc.common START=99 USE_PROCD=1 PROG=/bin/flood1.lua start_service() { config_load cgminer procd_open_instance procd_set_param command "$PROG" procd_set_param respawn ${respawn_threshold:-3600} ${respawn_timeout:-5} ${respawn_retry:-0} procd_set_param stdout 1 procd_set_param stderr 1 procd_close_instance } stop_service() { echo stop } service_triggers() { procd_add_reload_trigger "flood" } From c4ea27994bfa81e38dff9ccf9a92a33b5c612407 Mon Sep 17 00:00:00 2001 From: Jakub Horak Date: Thu, 11 Jan 2018 16:47:50 +0100 Subject: [PATCH] procd: Fix behavior when parsing long lines Processing of service stdout/stderr is silently stopped when long lines (>4096 bytes) are encountered. Moreover, when NUL byte (0) is received on input, the processing is stopped as well. When reading long lines procd waits until a newline character is received even though receive buffer is full. At that moment the file-descriptor is removed from epoll and communication stopson that descriptor stops. Similar thing happens with NUL byte, only that the newline is not found because "NUL" stops strchr() from processing further down the string. This patch corrects the behavior: long lines are broken into smaller ones with forced newline and NUL bytes are skipped when searching for a newline. Signed-off-by: Jakub Horak --- service/instance.c | 26 ++++++++++++++++++-------- 1 file changed, 18 insertions(+), 8 deletions(-) diff --git a/service/instance.c b/service/instance.c index a968a0b..0293099 100644 --- a/service/instance.c +++ b/service/instance.c @@ -461,24 +461,34 @@ instance_stdio(struct ustream *s, int prio, struct service_instance *in) { char *newline, *str, *arg0, ident[32]; int len; + int buflen; arg0 = basename(blobmsg_data(blobmsg_data(in->command))); snprintf(ident, sizeof(ident), "%s[%d]", arg0, in->proc.pid); ulog_open(ULOG_SYSLOG, LOG_DAEMON, ident); do { - str = ustream_get_read_buf(s, NULL); + str = ustream_get_read_buf(s, &buflen); if (!str) break; - newline = strchr(str, '\n'); - if (!newline) - break; - - *newline = 0; + /* search for '\n', take into account NUL bytes */ + newline = memchr(str, '\n', buflen); + if (!newline) { + /* is there a room in buffer? */ + if (buflen < s->r.buffer_len) { + /* yes - bailout, newline may still come */ + break; + } else { + /* no - force newline */ + len = buflen; + } + } else { + *newline = 0; + len = newline + 1 - str; + } + /* "str" is NUL terminated by ustream_get_read_buf */ ulog(prio, "%s\n", str); - - len = newline + 1 - str; ustream_consume(s, len); } while (1); -- 2.7.4