From patchwork Fri Aug 4 17:27:34 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yuxuan Luo X-Patchwork-Id: 1817040 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=lists.ubuntu.com (client-ip=91.189.94.19; helo=huckleberry.canonical.com; envelope-from=kernel-team-bounces@lists.ubuntu.com; receiver=) Authentication-Results: legolas.ozlabs.org; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=canonical.com header.i=@canonical.com header.a=rsa-sha256 header.s=20210705 header.b=J7lkNIET; dkim-atps=neutral Received: from huckleberry.canonical.com (huckleberry.canonical.com [91.189.94.19]) (using TLSv1.2 with cipher ECDHE-ECDSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4RHXjL0W58z1yYC for ; Sat, 5 Aug 2023 03:27:49 +1000 (AEST) Received: from localhost ([127.0.0.1] helo=huckleberry.canonical.com) by huckleberry.canonical.com with esmtp (Exim 4.86_2) (envelope-from ) id 1qRyaU-0005iY-Ed; Fri, 04 Aug 2023 17:27:42 +0000 Received: from smtp-relay-internal-1.internal ([10.131.114.114] helo=smtp-relay-internal-1.canonical.com) by huckleberry.canonical.com with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.86_2) (envelope-from ) id 1qRyaS-0005fu-OB for kernel-team@lists.ubuntu.com; Fri, 04 Aug 2023 17:27:40 +0000 Received: from mail-vk1-f198.google.com (mail-vk1-f198.google.com [209.85.221.198]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by smtp-relay-internal-1.canonical.com (Postfix) with ESMTPS id 5D03F3F205 for ; Fri, 4 Aug 2023 17:27:40 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=canonical.com; s=20210705; t=1691170060; bh=Cg0ZceWOShrvBoul4etokfqF1N40ueOhECbtmOXjZXA=; h=From:To:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=J7lkNIETJnSTqro6TGynTTijRO2gqCMn6YcLZujVsr0LkNvLoF7eDZVasF1LR4EZy xHceHEkWH9eHj478HD9etN+7ICUI2M2KBa+/5q0PRyq/4pgglGcSKmKhD/pv6cC4Ub HqJ5QO6pfr8pa3P0ppE1QTDUYbInLpvDNwOxJXemLUbLdpbm574AkuxCo+CKMJQG0q ohDYrhVJ8c0+CopmI28F/WD+bTWvcIJ+oiWiZKnN9sBdgvXXJ8J4yG+JC5wGO2rIP9 eHdvzC0baaKO4XRoCNTAFHByJYDojUpGAiTlyN9hCXD7a8fWHZQwLa2DIJdtZfi1hZ PyuC7l/ThGqGw== Received: by mail-vk1-f198.google.com with SMTP id 71dfb90a1353d-48642ecd129so598544e0c.0 for ; Fri, 04 Aug 2023 10:27:40 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1691170059; x=1691774859; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=Cg0ZceWOShrvBoul4etokfqF1N40ueOhECbtmOXjZXA=; b=EuARzSg2ip9qmMiZ43mEndjzlBihDFNqVizMn7zsgdFsgRELfl6pWRWc8UqryfNO0A zdKKkzE/bGtxtwI3Rlx1yz2bgVHWpb9+FuPEpW30ZwKvtxK1jgZfbh9HmYeMTnYfrwK9 4Z9YyggpbU0T7MciupvhfzPgavw5Dovsi1T+zQdsKzKHuUxvIBUk3c+BEvCo6U92bvXR kDBoUC1b1WyODjA7QESBvv7+DaqOu5n+CdnnetUdMHMpO9LbbukosVkVdeyWvHfSJdpV MeM6fJMdELQrmJg4WJ8LgFyifxo5DxRthHnPqRNcnbBcR0eZv0W64AJig6EkS6ek6Abc MmZg== X-Gm-Message-State: AOJu0Yy2cWPx7lL7TedBZsJpsnliZ3KaKw4Kj41EqFYJwOdpvF/isxIJ 3VAozs8vOWd999Ukke1oUzlg55q1LPWoVsAFA7MoDeDwSVZiNWlUcv3rhvlur89c5Q1gmhjd7rj MTHl1LfFzTkbdpSfwvBRRoH+Rpuhq3/5B+RDAmKrj14LxBjZjJg== X-Received: by 2002:a1f:c446:0:b0:487:1774:405e with SMTP id u67-20020a1fc446000000b004871774405emr1540212vkf.11.1691170058972; Fri, 04 Aug 2023 10:27:38 -0700 (PDT) X-Google-Smtp-Source: AGHT+IGyiCqXDoX8WX1mjyUjkzx5q1tUmhJsfNJjIxFUotK6u6MRtAJhh0bZ13d+HI0g2HwfhYUhtg== X-Received: by 2002:a1f:c446:0:b0:487:1774:405e with SMTP id u67-20020a1fc446000000b004871774405emr1540204vkf.11.1691170058660; Fri, 04 Aug 2023 10:27:38 -0700 (PDT) Received: from cache-ubuntu.hsd1.nj.comcast.net ([2601:86:200:98b0:a4fb:40c7:e7ca:9294]) by smtp.gmail.com with ESMTPSA id q18-20020a0ce212000000b0062dd9254429sm790461qvl.142.2023.08.04.10.27.38 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 04 Aug 2023 10:27:38 -0700 (PDT) From: Yuxuan Luo To: kernel-team@lists.ubuntu.com Subject: [SRU][Jammy-OEM-5.17/OEM-6.0][PATCH 1/2] net/sched: act_mirred: better wording on protection against excessive stack growth Date: Fri, 4 Aug 2023 13:27:34 -0400 Message-Id: <20230804172735.20929-2-yuxuan.luo@canonical.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230804172735.20929-1-yuxuan.luo@canonical.com> References: <20230804172735.20929-1-yuxuan.luo@canonical.com> MIME-Version: 1.0 X-BeenThere: kernel-team@lists.ubuntu.com X-Mailman-Version: 2.1.20 Precedence: list List-Id: Kernel team discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: kernel-team-bounces@lists.ubuntu.com Sender: "kernel-team" From: Davide Caratti with commit e2ca070f89ec ("net: sched: protect against stack overflow in TC act_mirred"), act_mirred protected itself against excessive stack growth using per_cpu counter of nested calls to tcf_mirred_act(), and capping it to MIRRED_RECURSION_LIMIT. However, such protection does not detect recursion/loops in case the packet is enqueued to the backlog (for example, when the mirred target device has RPS or skb timestamping enabled). Change the wording from "recursion" to "nesting" to make it more clear to readers. CC: Jamal Hadi Salim Signed-off-by: Davide Caratti Reviewed-by: Marcelo Ricardo Leitner Acked-by: Jamal Hadi Salim Signed-off-by: Paolo Abeni (cherry picked from commit 78dcdffe0418ac8f3f057f26fe71ccf4d8ed851f) CVE-2022-4269 Signed-off-by: Yuxuan Luo --- net/sched/act_mirred.c | 16 ++++++++-------- 1 file changed, 8 insertions(+), 8 deletions(-) diff --git a/net/sched/act_mirred.c b/net/sched/act_mirred.c index a1d70cf86843..914abe962da7 100644 --- a/net/sched/act_mirred.c +++ b/net/sched/act_mirred.c @@ -28,8 +28,8 @@ static LIST_HEAD(mirred_list); static DEFINE_SPINLOCK(mirred_list_lock); -#define MIRRED_RECURSION_LIMIT 4 -static DEFINE_PER_CPU(unsigned int, mirred_rec_level); +#define MIRRED_NEST_LIMIT 4 +static DEFINE_PER_CPU(unsigned int, mirred_nest_level); static bool tcf_mirred_is_act_redirect(int action) { @@ -225,7 +225,7 @@ static int tcf_mirred_act(struct sk_buff *skb, const struct tc_action *a, struct sk_buff *skb2 = skb; bool m_mac_header_xmit; struct net_device *dev; - unsigned int rec_level; + unsigned int nest_level; int retval, err = 0; bool use_reinsert; bool want_ingress; @@ -236,11 +236,11 @@ static int tcf_mirred_act(struct sk_buff *skb, const struct tc_action *a, int mac_len; bool at_nh; - rec_level = __this_cpu_inc_return(mirred_rec_level); - if (unlikely(rec_level > MIRRED_RECURSION_LIMIT)) { + nest_level = __this_cpu_inc_return(mirred_nest_level); + if (unlikely(nest_level > MIRRED_NEST_LIMIT)) { net_warn_ratelimited("Packet exceeded mirred recursion limit on dev %s\n", netdev_name(skb->dev)); - __this_cpu_dec(mirred_rec_level); + __this_cpu_dec(mirred_nest_level); return TC_ACT_SHOT; } @@ -310,7 +310,7 @@ static int tcf_mirred_act(struct sk_buff *skb, const struct tc_action *a, err = tcf_mirred_forward(res->ingress, skb); if (err) tcf_action_inc_overlimit_qstats(&m->common); - __this_cpu_dec(mirred_rec_level); + __this_cpu_dec(mirred_nest_level); return TC_ACT_CONSUMED; } } @@ -322,7 +322,7 @@ static int tcf_mirred_act(struct sk_buff *skb, const struct tc_action *a, if (tcf_mirred_is_act_redirect(m_eaction)) retval = TC_ACT_SHOT; } - __this_cpu_dec(mirred_rec_level); + __this_cpu_dec(mirred_nest_level); return retval; } From patchwork Fri Aug 4 17:27:35 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yuxuan Luo X-Patchwork-Id: 1817041 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=lists.ubuntu.com (client-ip=91.189.94.19; helo=huckleberry.canonical.com; envelope-from=kernel-team-bounces@lists.ubuntu.com; receiver=) Authentication-Results: legolas.ozlabs.org; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=canonical.com header.i=@canonical.com header.a=rsa-sha256 header.s=20210705 header.b=DULIVENQ; dkim-atps=neutral Received: from huckleberry.canonical.com (huckleberry.canonical.com [91.189.94.19]) (using TLSv1.2 with cipher ECDHE-ECDSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4RHXjL1wk4z1yds for ; Sat, 5 Aug 2023 03:27:50 +1000 (AEST) Received: from localhost ([127.0.0.1] helo=huckleberry.canonical.com) by huckleberry.canonical.com with esmtp (Exim 4.86_2) (envelope-from ) id 1qRyaV-0005j3-Kl; Fri, 04 Aug 2023 17:27:43 +0000 Received: from smtp-relay-internal-0.internal ([10.131.114.225] helo=smtp-relay-internal-0.canonical.com) by huckleberry.canonical.com with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.86_2) (envelope-from ) id 1qRyaT-0005hF-K2 for kernel-team@lists.ubuntu.com; Fri, 04 Aug 2023 17:27:41 +0000 Received: from mail-vs1-f70.google.com (mail-vs1-f70.google.com [209.85.217.70]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by smtp-relay-internal-0.canonical.com (Postfix) with ESMTPS id 1C1C43F077 for ; Fri, 4 Aug 2023 17:27:41 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=canonical.com; s=20210705; t=1691170061; bh=BdUoNh6AyV++WdjCy2zyE0YQQIHmv06R87foB+e9fg0=; h=From:To:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=DULIVENQQ3cFSSXBsxmq8PYraNnantZumCZgjxkpyFT2qb9f0jTZRnuRmSikvIVuf Ezpn8x4jHk3z4jw/RJpOvqLy1+ngpIm18ee1EURN4mZlx5Y3efwtjw65qrhf2bGKKK R3faksI/0qwsFsxYQLf8DkNmd6DZHjj6j8urhGz9ffHPjh/asv3mBtLL6LGwj9W91X JKyDwloNioxFkKnFIIECJRg/a91DO5LIwh/QjuaqMte0gUphI+gHDu/omTfh4cX22q U5DeX5TYF2DQmdy39DtcdfIcnCdAch5FPw03YBoiEwi+/CRZReuRvU+64yvysV1TFx DvNOhr4OtqJ7w== Received: by mail-vs1-f70.google.com with SMTP id ada2fe7eead31-44782d3974fso603403137.3 for ; Fri, 04 Aug 2023 10:27:41 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1691170060; x=1691774860; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=BdUoNh6AyV++WdjCy2zyE0YQQIHmv06R87foB+e9fg0=; b=BzZGe62s5sWyL1kn83J/b+i2bob30NlKRLNIkYntd53R4eWLeiAN2yHbcufbF3rL1T eQlW+BZB10AuDw9FthpL3bTfxzRq3WxO5wjoIxpMQTlF0i6jF405xah3pjPK0EnuEVUJ 3n0BnyMPNgELLoLuUAGE47P+46c6lja84v5fK6dMQlqPNRwgHQpMnQmsGYLbk+R70oyJ BYwXRTtHsvMnXiQ6NkZDJ9swNaN5C+3MTiwiaqMy5W+47f52BfSuiFztUllUTQdCW+0B nlLbreZaBfAhtqMYJ6Oy9r+BpnqMJyK3IagIwgAtjq37jfrwVbzkWBtCF1B04jux3RXw K9mw== X-Gm-Message-State: AOJu0YyKBXd/DhuZfVxebd1pAjX63E5kAx6ezNOn/6saETWx9wWHLP23 1oSg5gSDYhyClSx4WQ35uW+RXlx0WA4rVJgPZJocOf+1xnguqK/a4D8OK5+DoDPIYwDRkC8mswC O9rvuyuKx0QRPJL8xNYhiP9PPRz0tbypCJdXOFe6JQoyvZQ6dDQ== X-Received: by 2002:a67:fc03:0:b0:445:4996:1d27 with SMTP id o3-20020a67fc03000000b0044549961d27mr1808779vsq.3.1691170059733; Fri, 04 Aug 2023 10:27:39 -0700 (PDT) X-Google-Smtp-Source: AGHT+IFYdfpeJvGw1UtRuSoLJTkFSrSogW0zM9LO3IvEFFxVCZPxVPSK00IZ2mGcbeIFET/uyWMp2A== X-Received: by 2002:a67:fc03:0:b0:445:4996:1d27 with SMTP id o3-20020a67fc03000000b0044549961d27mr1808762vsq.3.1691170059381; Fri, 04 Aug 2023 10:27:39 -0700 (PDT) Received: from cache-ubuntu.hsd1.nj.comcast.net ([2601:86:200:98b0:a4fb:40c7:e7ca:9294]) by smtp.gmail.com with ESMTPSA id q18-20020a0ce212000000b0062dd9254429sm790461qvl.142.2023.08.04.10.27.38 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 04 Aug 2023 10:27:39 -0700 (PDT) From: Yuxuan Luo To: kernel-team@lists.ubuntu.com Subject: [SRU][Jammy-OEM-5.17/OEM-6.0][PATCH 2/2] act_mirred: use the backlog for nested calls to mirred ingress Date: Fri, 4 Aug 2023 13:27:35 -0400 Message-Id: <20230804172735.20929-3-yuxuan.luo@canonical.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230804172735.20929-1-yuxuan.luo@canonical.com> References: <20230804172735.20929-1-yuxuan.luo@canonical.com> MIME-Version: 1.0 X-BeenThere: kernel-team@lists.ubuntu.com X-Mailman-Version: 2.1.20 Precedence: list List-Id: Kernel team discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: kernel-team-bounces@lists.ubuntu.com Sender: "kernel-team" From: Davide Caratti William reports kernel soft-lockups on some OVS topologies when TC mirred egress->ingress action is hit by local TCP traffic [1]. The same can also be reproduced with SCTP (thanks Xin for verifying), when client and server reach themselves through mirred egress to ingress, and one of the two peers sends a "heartbeat" packet (from within a timer). Enqueueing to backlog proved to fix this soft lockup; however, as Cong noticed [2], we should preserve - when possible - the current mirred behavior that counts as "overlimits" any eventual packet drop subsequent to the mirred forwarding action [3]. A compromise solution might use the backlog only when tcf_mirred_act() has a nest level greater than one: change tcf_mirred_forward() accordingly. Also, add a kselftest that can reproduce the lockup and verifies TC mirred ability to account for further packet drops after TC mirred egress->ingress (when the nest level is 1). [1] https://lore.kernel.org/netdev/33dc43f587ec1388ba456b4915c75f02a8aae226.1663945716.git.dcaratti@redhat.com/ [2] https://lore.kernel.org/netdev/Y0w%2FWWY60gqrtGLp@pop-os.localdomain/ [3] such behavior is not guaranteed: for example, if RPS or skb RX timestamping is enabled on the mirred target device, the kernel can defer receiving the skb and return NET_RX_SUCCESS inside tcf_mirred_forward(). Reported-by: William Zhao CC: Xin Long Signed-off-by: Davide Caratti Reviewed-by: Marcelo Ricardo Leitner Acked-by: Jamal Hadi Salim Signed-off-by: Paolo Abeni (cherry picked from commit ca22da2fbd693b54dc8e3b7b54ccc9f7e9ba3640) CVE-2022-4269 Signed-off-by: Yuxuan Luo --- net/sched/act_mirred.c | 7 +++ .../selftests/net/forwarding/tc_actions.sh | 49 ++++++++++++++++++- 2 files changed, 55 insertions(+), 1 deletion(-) diff --git a/net/sched/act_mirred.c b/net/sched/act_mirred.c index 914abe962da7..5a107fc8c743 100644 --- a/net/sched/act_mirred.c +++ b/net/sched/act_mirred.c @@ -206,12 +206,19 @@ static int tcf_mirred_init(struct net *net, struct nlattr *nla, return err; } +static bool is_mirred_nested(void) +{ + return unlikely(__this_cpu_read(mirred_nest_level) > 1); +} + static int tcf_mirred_forward(bool want_ingress, struct sk_buff *skb) { int err; if (!want_ingress) err = tcf_dev_queue_xmit(skb, dev_queue_xmit); + else if (is_mirred_nested()) + err = netif_rx(skb); else err = netif_receive_skb(skb); diff --git a/tools/testing/selftests/net/forwarding/tc_actions.sh b/tools/testing/selftests/net/forwarding/tc_actions.sh index 1e0a62f638fe..919c0dd9fe4b 100755 --- a/tools/testing/selftests/net/forwarding/tc_actions.sh +++ b/tools/testing/selftests/net/forwarding/tc_actions.sh @@ -3,7 +3,8 @@ ALL_TESTS="gact_drop_and_ok_test mirred_egress_redirect_test \ mirred_egress_mirror_test matchall_mirred_egress_mirror_test \ - gact_trap_test mirred_egress_to_ingress_test" + gact_trap_test mirred_egress_to_ingress_test \ + mirred_egress_to_ingress_tcp_test" NUM_NETIFS=4 source tc_common.sh source lib.sh @@ -198,6 +199,52 @@ mirred_egress_to_ingress_test() log_test "mirred_egress_to_ingress ($tcflags)" } +mirred_egress_to_ingress_tcp_test() +{ + local tmpfile=$(mktemp) tmpfile1=$(mktemp) + + RET=0 + dd conv=sparse status=none if=/dev/zero bs=1M count=2 of=$tmpfile + tc filter add dev $h1 protocol ip pref 100 handle 100 egress flower \ + $tcflags ip_proto tcp src_ip 192.0.2.1 dst_ip 192.0.2.2 \ + action ct commit nat src addr 192.0.2.2 pipe \ + action ct clear pipe \ + action ct commit nat dst addr 192.0.2.1 pipe \ + action ct clear pipe \ + action skbedit ptype host pipe \ + action mirred ingress redirect dev $h1 + tc filter add dev $h1 protocol ip pref 101 handle 101 egress flower \ + $tcflags ip_proto icmp \ + action mirred ingress redirect dev $h1 + tc filter add dev $h1 protocol ip pref 102 handle 102 ingress flower \ + ip_proto icmp \ + action drop + + ip vrf exec v$h1 nc --recv-only -w10 -l -p 12345 -o $tmpfile1 & + local rpid=$! + ip vrf exec v$h1 nc -w1 --send-only 192.0.2.2 12345 <$tmpfile + wait -n $rpid + cmp -s $tmpfile $tmpfile1 + check_err $? "server output check failed" + + $MZ $h1 -c 10 -p 64 -a $h1mac -b $h1mac -A 192.0.2.1 -B 192.0.2.1 \ + -t icmp "ping,id=42,seq=5" -q + tc_check_packets "dev $h1 egress" 101 10 + check_err $? "didn't mirred redirect ICMP" + tc_check_packets "dev $h1 ingress" 102 10 + check_err $? "didn't drop mirred ICMP" + local overlimits=$(tc_rule_stats_get ${h1} 101 egress .overlimits) + test ${overlimits} = 10 + check_err $? "wrong overlimits, expected 10 got ${overlimits}" + + tc filter del dev $h1 egress protocol ip pref 100 handle 100 flower + tc filter del dev $h1 egress protocol ip pref 101 handle 101 flower + tc filter del dev $h1 ingress protocol ip pref 102 handle 102 flower + + rm -f $tmpfile $tmpfile1 + log_test "mirred_egress_to_ingress_tcp ($tcflags)" +} + setup_prepare() { h1=${NETIFS[p1]}