From patchwork Mon Jan 23 19:06:53 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dan Brown X-Patchwork-Id: 1730591 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; spf=none (no SPF record) smtp.mailfrom=lists.openwrt.org (client-ip=2607:7c80:54:3::133; helo=bombadil.infradead.org; envelope-from=openwrt-devel-bounces+incoming=patchwork.ozlabs.org@lists.openwrt.org; receiver=) Authentication-Results: legolas.ozlabs.org; dkim=pass (2048-bit key; secure) header.d=lists.infradead.org header.i=@lists.infradead.org header.a=rsa-sha256 header.s=bombadil.20210309 header.b=v3kUUgIF; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.a=rsa-sha256 header.s=20210112 header.b=lekWNu+k; dkim-atps=neutral Received: from bombadil.infradead.org (bombadil.infradead.org [IPv6:2607:7c80:54:3::133]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-384) server-digest SHA384) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4P10796ZYRz23gd for ; Tue, 24 Jan 2023 06:10:45 +1100 (AEDT) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:To:Subject:Message-ID:Date:From: In-Reply-To:References:MIME-Version:Reply-To:Cc:Content-ID: Content-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc :Resent-Message-ID:List-Owner; bh=35szFabpdCz9dDxPOWV6lsZbN7sUxWMGUlEVftp5FkI=; b=v3kUUgIFpiqAaWp8YY7Hb3wUMz XAXbXUlqQXta5mTdnNmKC0oJMNJSGz+jAgku2dS8b1Q6ujvWJprV6qS8chuhYPkGX/guHqe2WuHZk Ts1TPWyVrVUxaV9Gg+we1UhK+R063N4W0wt7q+Pu7gUagdcSvcG9p2J9j2d8Qf3YJx7F0tWmI5qqD BUbk0nytpT1QhCyW3zesFm7mGXDTfoEKEsvkXbvf42XRweWwemuxclrc2ehCYTcpTjRw5j3jZ6IS5 JFjWqQo3GaCQAt28K0pv2+eA4vepdJoJyA8K6/pJgBpVUXxZwzclQF/9XnlChmYICLylAnIW4ZFGP JQufUYRw==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1pK2A5-0016O0-1n; Mon, 23 Jan 2023 19:07:21 +0000 Received: from mail-qv1-xf35.google.com ([2607:f8b0:4864:20::f35]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1pK29y-0016Kr-WA for openwrt-devel@lists.openwrt.org; Mon, 23 Jan 2023 19:07:16 +0000 Received: by mail-qv1-xf35.google.com with SMTP id g10so9767695qvo.6 for ; Mon, 23 Jan 2023 11:07:06 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=to:subject:message-id:date:from:in-reply-to:references:mime-version :from:to:cc:subject:date:message-id:reply-to; bh=Eo54sWJ4EahpikPhj1qVhnYXWQD3jOFDYYNvE3lz9vQ=; b=lekWNu+k3K3YJ1LWlViWD07vILHIFGEOqeqdWyaJiYaM65NXno7a/xpYhz6S7sF4ne 3N3hXpIOit+LiRcCpbHfM1YrKsUffHVNBPlLtkV54OvDyzD4ZvfgTvwVFcQ5xAoGUlnk KNDkDIZl2qaCB4QHwYSidort7mdqcIyMcXfKirUNZJ8pm/p+Hvr5YDSzAyt1p7N9I3Z8 MZiSN4OFTAIexY9DPdbS+YlOmbFBi+1l2MOcK7VhH2zRcdRZHEkosRT05bjA1KzsjZoK RjER3H2DxKFOyY9tXN6iC7G4uHCEdkpOU8Yxz3uIaICpQdRMVoYrM6KOB/FN2q+Ye+vq 4EdQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=to:subject:message-id:date:from:in-reply-to:references:mime-version :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=Eo54sWJ4EahpikPhj1qVhnYXWQD3jOFDYYNvE3lz9vQ=; b=AjAfU7B0s7989EuSqGwhzdLmZskyTGBgT31lmwJaybqU+pETdKlhA+Mn7po1moSq01 hLWj4JjSJ00W5QzF4g99W5/h1TDwb5or63+HBmSvCK25IynoqWTjLcF8JOtIv9p1RIN+ OuynzIko0zpbnvVaYXls0Rzs3ws0MV+tDcijn/qRrAfntGlpXQNBevi9quwihsBgLtCa +EOisS0+wNOGKEuXUrBDmGQRXiqZM7PR/HzfOOQKJX5O1LrK/8aLe38XGefrthGM+wUm 2awVIZLHm6F+eOPhaJSq9FL1f+c56UFPqxaIAXwWP7JjRH0EV7VEwbCAbiNf+Xrt4HUA 9Fuw== X-Gm-Message-State: AFqh2kqtJidV9pdnOM6ERcdU/S9Av9qn9a8wTJWtesjfGRuoBROB9hXP kUeOYQXW/qIfLq/0dT18lMxZVJ4eYnS0k8QmnuDXKrhKtIqoSA== X-Google-Smtp-Source: AMrXdXt+asSWrEx3IXy1C0dNzqB6DJcPl4w6MXkkW2pFY/6JlFGE1UIooBbOemPb6bAll0kAlm3y7cY4+Tm/1jUWuck= X-Received: by 2002:ad4:530e:0:b0:535:276:9240 with SMTP id y14-20020ad4530e000000b0053502769240mr1347831qvr.37.1674500825306; Mon, 23 Jan 2023 11:07:05 -0800 (PST) MIME-Version: 1.0 References: In-Reply-To: From: Dan Brown Date: Mon, 23 Jan 2023 20:06:53 +0100 Message-ID: Subject: [patch] logread : add option to exclude lines matching regex To: openwrt-devel@lists.openwrt.org X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20230123_110715_095612_8AA6392E X-CRM114-Status: GOOD ( 16.83 ) X-Spam-Score: -0.2 (/) X-Spam-Report: Spam detection software, running on the system "bombadil.infradead.org", has NOT identified this incoming email as spam. The original message has been attached to this so you can view it or label similar future email. If you have any questions, see the administrator of that system for details. Content preview: From: Dan Brown logread: add option to exclude lines matching regex This adds another filtering option (-E) to exclude lines that match a regex, similar to the existing include option (-e): -e include messages matching a regexp -E exclude messages [...] Content analysis details: (-0.2 points, 5.0 required) pts rule name description ---- ---------------------- -------------------------------------------------- -0.0 RCVD_IN_DNSWL_NONE RBL: Sender listed at https://www.dnswl.org/, no trust [2607:f8b0:4864:20:0:0:0:f35 listed in] [list.dnswl.org] -0.0 SPF_PASS SPF: sender matches SPF record 0.0 SPF_HELO_NONE SPF: HELO does not publish an SPF Record 0.0 FREEMAIL_FROM Sender email is commonly abused enduser mail provider [danbrown[at]gmail.com] -0.1 DKIM_VALID_AU Message has a valid DKIM or DK signature from author's domain -0.1 DKIM_VALID Message has at least one valid DKIM or DK signature 0.1 DKIM_SIGNED Message has a DKIM or DK signature, not necessarily valid -0.1 DKIM_VALID_EF Message has a valid DKIM or DK signature from envelope-from domain X-BeenThere: openwrt-devel@lists.openwrt.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: OpenWrt Development List List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "openwrt-devel" Errors-To: openwrt-devel-bounces+incoming=patchwork.ozlabs.org@lists.openwrt.org From: Dan Brown logread: add option to exclude lines matching regex This adds another filtering option (-E) to exclude lines that match a regex, similar to the existing include option (-e): -e include messages matching a regexp -E exclude messages matching a regexp The two options -e and -E are independent, and lines must pass both tests to appear in program output. * 'logread -E dhcp' will display all lines that do not contain the string "dhcp" * 'logread -e dhcp -E 192.168' will display all lines that contain "dhcp" and do not contain "192.168" signed off by: Dan Brown, danbrown@gmail.com --- Further update: Sorry for yet another (third) version of this. I discovered that the regex match checked only the contents of the message part of the log, but I want to match the entire formatted logline with program, priority, etc. This new version does that. This probably deserves some discussion because it changes the behavior of the existing '-e' option, and may not be what some people want to match on. Are there use cases that depend on matching only the message part of the log line? Update: this newer (second) version fixes a bug where the -e and -E options would be mutually exclusive instead of independent. Sorry. The motivation for adding this option to logread is to remove frequent and uninteresting loglines from being displayed in Luci. One way to achieve this is to install an alternate logging system (syslog-ng or rsyslog) and set up rules for filtering there. Instead of that, I would prefer to use the simpler and standard logd/ubox method with this small enhancement. This patch applies to the ubox project (https://git.openwrt.org/?p=project/ubox.git;a=summary) static int log_timestamp; @@ -157,9 +158,6 @@ static int log_notify(struct blob_attr * return 0; m = blobmsg_get_string(tb[LOG_MSG]); - if (regexp_pattern && - regexec(®exp_preg, m, 0, NULL, 0) == REG_NOMATCH) - return 0; t = blobmsg_get_u64(tb[LOG_TIME]) / 1000; if (log_timestamp) { t_ms = blobmsg_get_u64(tb[LOG_TIME]) % 1000; @@ -188,6 +186,12 @@ static int log_notify(struct blob_attr * if (blobmsg_get_u32(tb[LOG_SOURCE]) == SOURCE_KLOG) strncat(buf, "kernel: ", sizeof(buf) - strlen(buf) - 1); strncat(buf, m, sizeof(buf) - strlen(buf) - 1); + if (regexp_pattern_include && + regexec(®exp_preg_include, buf, 0, NULL, 0) == REG_NOMATCH) + return 0; + if (regexp_pattern_exclude && + regexec(®exp_preg_exclude, buf, 0, NULL, 0) != REG_NOMATCH) + return 0; if (log_udp) err = write(sender.fd, buf, strlen(buf)); else { @@ -211,6 +215,12 @@ static int log_notify(struct blob_attr * getcodetext(LOG_FAC(p) << 3, facilitynames), getcodetext(LOG_PRI(p), prioritynames), (blobmsg_get_u32(tb[LOG_SOURCE])) ? ("") : (" kernel:"), m); + if (regexp_pattern_include && + regexec(®exp_preg_include, buf, 0, NULL, 0) == REG_NOMATCH) + return 0; + if (regexp_pattern_exclude && + regexec(®exp_preg_exclude, buf, 0, NULL, 0) != REG_NOMATCH) + return 0; ret = write(sender.fd, buf, strlen(buf)); } @@ -226,7 +236,8 @@ static int usage(const char *prog) "Options:\n" " -s Path to ubus socket\n" " -l Got only the last 'count' messages\n" - " -e Filter messages with a regexp\n" + " -e include log lines that match a regexp\n" + " -E exclude log lines that match a regexp\n" " -r Stream message to a server\n" " -F Log file\n" " -S Log size\n" @@ -313,7 +324,7 @@ int main(int argc, char **argv) signal(SIGPIPE, SIG_IGN); - while ((ch = getopt(argc, argv, "u0fcs:l:z:Z:r:F:p:S:P:h:e:t")) != -1) { + while ((ch = getopt(argc, argv, "u0fcs:l:z:Z:r:F:p:S:P:h:e:E:t")) != -1) { switch (ch) { case 'u': log_udp = 1; @@ -361,8 +372,13 @@ int main(int argc, char **argv) hostname = optarg; break; case 'e': - if (!regcomp(®exp_preg, optarg, REG_NOSUB)) { - regexp_pattern = optarg; + if (!regcomp(®exp_preg_include, optarg, REG_NOSUB)) { + regexp_pattern_include = optarg; + } + break; + case 'E': + if (!regcomp(®exp_preg_exclude, optarg, REG_NOSUB)) { + regexp_pattern_exclude = optarg; } break; case 't': --- log/logread.c.orig 2023-01-23 13:16:59.502476137 +0100 +++ log/logread.c 2023-01-23 20:05:48.478603378 +0100 @@ -60,8 +60,9 @@ static const struct blobmsg_policy log_p static struct uloop_timeout retry; static struct uloop_fd sender; -static regex_t regexp_preg; -static const char *log_file, *log_ip, *log_port, *log_prefix, *pid_file, *hostname, *regexp_pattern; +static regex_t regexp_preg_include, regexp_preg_exclude; +static const char *log_file, *log_ip, *log_port, *log_prefix, *pid_file; +static const char *hostname, *regexp_pattern_include, *regexp_pattern_exclude; static int log_type = LOG_STDOUT; static int log_size, log_udp, log_follow, log_trailer_null = 0;