From patchwork Tue May 9 23:50:39 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yuxuan Luo X-Patchwork-Id: 1779141 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=lists.ubuntu.com (client-ip=91.189.94.19; helo=huckleberry.canonical.com; envelope-from=kernel-team-bounces@lists.ubuntu.com; receiver=) Authentication-Results: legolas.ozlabs.org; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=canonical.com header.i=@canonical.com header.a=rsa-sha256 header.s=20210705 header.b=l6XShk3d; dkim-atps=neutral Received: from huckleberry.canonical.com (huckleberry.canonical.com [91.189.94.19]) (using TLSv1.2 with cipher ECDHE-ECDSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4QGFLG6H85z214j for ; Wed, 10 May 2023 09:51:34 +1000 (AEST) Received: from localhost ([127.0.0.1] helo=huckleberry.canonical.com) by huckleberry.canonical.com with esmtp (Exim 4.86_2) (envelope-from ) id 1pwX78-0007Pf-KC; Tue, 09 May 2023 23:51:26 +0000 Received: from smtp-relay-internal-1.internal ([10.131.114.114] helo=smtp-relay-internal-1.canonical.com) by huckleberry.canonical.com with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.86_2) (envelope-from ) id 1pwX76-0007P1-0l for kernel-team@lists.ubuntu.com; Tue, 09 May 2023 23:51:24 +0000 Received: from mail-ed1-f70.google.com (mail-ed1-f70.google.com [209.85.208.70]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by smtp-relay-internal-1.canonical.com (Postfix) with ESMTPS id C6D7E3F486 for ; Tue, 9 May 2023 23:51:23 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=canonical.com; s=20210705; t=1683676283; bh=c+x8ZTQFJU0PvxcftK1GIjESgMSJygxsjvV2CJQtw08=; h=From:To:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=l6XShk3d7Ni3urInFEz5CvQOUyUU0DU1vGyGYRSZ4Dls/Y82L2ip3m/y+bCB5ykMg hutTUXiKw7KWHh9UKBVTgnGnYyzSjZqKJUaTbdUFoUIP/w5KST5aSs2HmltQGHOodB JVA+SYvNcGNybuZErTrFV0W1ZYMdvXyalYekzY/D7AmZjgDn2W/kl1XdsydyOZIXG/ PFznC5p9S0V8rjT6kkVsPL7+m4w8PVNULGapszlfs+X+IMhjLLlzhbs9HhB4Xv+F36 PC9WotiUxtl6pcVqA58wWCpDDewFm4BIrKElZT8WCxUEsGyFkS/fyC+9mOHWcFQU1A rLeWmALZARWhQ== Received: by mail-ed1-f70.google.com with SMTP id 4fb4d7f45d1cf-50bc9da53a3so7653982a12.1 for ; Tue, 09 May 2023 16:51:23 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1683676283; x=1686268283; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=c+x8ZTQFJU0PvxcftK1GIjESgMSJygxsjvV2CJQtw08=; b=PreScSWEbIEHHmUaj3AJbD/dVOk2pe4XABJsP+ddxYoYN+kJSeRMLLg1ovIp8acOYS auN5MEJE2qarorRw9gnej/ljFZmOWiOypR/TeecnxbIVQZ/iHgINy0nfiNO/6OJqRWtu fTdcddYIIlJd/stGXfy+EuXUNeD8xdOnex6IIftWyHUt3MyfW7Al4y/TBwHJQoNaK2QD SSMDweEoTM1QGngCXei9fdwd93hJ9gl/1f+sXAVx1nGaPLFgv6N6WdxkfaPFo5BuaD8c hDq2w+oInmlv7Cq2fJZbl9YoHYgcALlIVaxmE0753Yp6z7l0nquSTba8J9jq1sgaG81K 0FEA== X-Gm-Message-State: AC+VfDz8715Ph0CwkWJJzAW4JGiPuiR9DXIYOWhrmpVgYX17MybLvZyv svPmAHf5pFNiNBpXhk11KCGHT30oLQij38TLj6OVdaf182TFKfinCsZOZFvVy2uS2mbmbsiJrta GrrG5WeHfeVXbi3yE2c836c9JT+mtyOVus2WIf8GnT4BbnsfQSw== X-Received: by 2002:a05:6402:793:b0:506:bcd6:b125 with SMTP id d19-20020a056402079300b00506bcd6b125mr12062230edy.40.1683676283135; Tue, 09 May 2023 16:51:23 -0700 (PDT) X-Google-Smtp-Source: ACHHUZ5CKSKXfJ30ma0GAmukdEW/q/SokZC0OxtScnFawAC7vhj+zUHZtDQW0Myly8mkaSuttjAlRg== X-Received: by 2002:a05:6402:793:b0:506:bcd6:b125 with SMTP id d19-20020a056402079300b00506bcd6b125mr12062221edy.40.1683676282793; Tue, 09 May 2023 16:51:22 -0700 (PDT) Received: from localhost.localdomain ([2001:67c:1562:8007::aac:4795]) by smtp.gmail.com with ESMTPSA id p15-20020aa7cc8f000000b0050bcca2e459sm1295429edt.8.2023.05.09.16.51.21 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 09 May 2023 16:51:22 -0700 (PDT) From: Yuxuan Luo To: kernel-team@lists.ubuntu.com Subject: [SRU][Kinetic/Jammy][PATCH 1/2] act_mirred: use the backlog for nested calls to mirred ingress Date: Tue, 9 May 2023 19:50:39 -0400 Message-Id: <20230509235043.69974-2-yuxuan.luo@canonical.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230509235043.69974-1-yuxuan.luo@canonical.com> References: <20230509235043.69974-1-yuxuan.luo@canonical.com> MIME-Version: 1.0 X-BeenThere: kernel-team@lists.ubuntu.com X-Mailman-Version: 2.1.20 Precedence: list List-Id: Kernel team discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: kernel-team-bounces@lists.ubuntu.com Sender: "kernel-team" From: Davide Caratti William reports kernel soft-lockups on some OVS topologies when TC mirred egress->ingress action is hit by local TCP traffic [1]. The same can also be reproduced with SCTP (thanks Xin for verifying), when client and server reach themselves through mirred egress to ingress, and one of the two peers sends a "heartbeat" packet (from within a timer). Enqueueing to backlog proved to fix this soft lockup; however, as Cong noticed [2], we should preserve - when possible - the current mirred behavior that counts as "overlimits" any eventual packet drop subsequent to the mirred forwarding action [3]. A compromise solution might use the backlog only when tcf_mirred_act() has a nest level greater than one: change tcf_mirred_forward() accordingly. Also, add a kselftest that can reproduce the lockup and verifies TC mirred ability to account for further packet drops after TC mirred egress->ingress (when the nest level is 1). [1] https://lore.kernel.org/netdev/33dc43f587ec1388ba456b4915c75f02a8aae226.1663945716.git.dcaratti@redhat.com/ [2] https://lore.kernel.org/netdev/Y0w%2FWWY60gqrtGLp@pop-os.localdomain/ [3] such behavior is not guaranteed: for example, if RPS or skb RX timestamping is enabled on the mirred target device, the kernel can defer receiving the skb and return NET_RX_SUCCESS inside tcf_mirred_forward(). Reported-by: William Zhao CC: Xin Long Signed-off-by: Davide Caratti Reviewed-by: Marcelo Ricardo Leitner Acked-by: Jamal Hadi Salim Signed-off-by: Paolo Abeni (cherry picked from commit ca22da2fbd693b54dc8e3b7b54ccc9f7e9ba3640) CVE-2022-4269 Signed-off-by: Yuxuan Luo --- net/sched/act_mirred.c | 7 +++ .../selftests/net/forwarding/tc_actions.sh | 49 ++++++++++++++++++- 2 files changed, 55 insertions(+), 1 deletion(-) diff --git a/net/sched/act_mirred.c b/net/sched/act_mirred.c index ebb92fb072ab5..66f9078e7c087 100644 --- a/net/sched/act_mirred.c +++ b/net/sched/act_mirred.c @@ -206,12 +206,19 @@ static int tcf_mirred_init(struct net *net, struct nlattr *nla, return err; } +static bool is_mirred_nested(void) +{ + return unlikely(__this_cpu_read(mirred_nest_level) > 1); +} + static int tcf_mirred_forward(bool want_ingress, struct sk_buff *skb) { int err; if (!want_ingress) err = tcf_dev_queue_xmit(skb, dev_queue_xmit); + else if (is_mirred_nested()) + err = netif_rx(skb); else err = netif_receive_skb(skb); diff --git a/tools/testing/selftests/net/forwarding/tc_actions.sh b/tools/testing/selftests/net/forwarding/tc_actions.sh index 1e0a62f638fec..919c0dd9fe4bc 100755 --- a/tools/testing/selftests/net/forwarding/tc_actions.sh +++ b/tools/testing/selftests/net/forwarding/tc_actions.sh @@ -3,7 +3,8 @@ ALL_TESTS="gact_drop_and_ok_test mirred_egress_redirect_test \ mirred_egress_mirror_test matchall_mirred_egress_mirror_test \ - gact_trap_test mirred_egress_to_ingress_test" + gact_trap_test mirred_egress_to_ingress_test \ + mirred_egress_to_ingress_tcp_test" NUM_NETIFS=4 source tc_common.sh source lib.sh @@ -198,6 +199,52 @@ mirred_egress_to_ingress_test() log_test "mirred_egress_to_ingress ($tcflags)" } +mirred_egress_to_ingress_tcp_test() +{ + local tmpfile=$(mktemp) tmpfile1=$(mktemp) + + RET=0 + dd conv=sparse status=none if=/dev/zero bs=1M count=2 of=$tmpfile + tc filter add dev $h1 protocol ip pref 100 handle 100 egress flower \ + $tcflags ip_proto tcp src_ip 192.0.2.1 dst_ip 192.0.2.2 \ + action ct commit nat src addr 192.0.2.2 pipe \ + action ct clear pipe \ + action ct commit nat dst addr 192.0.2.1 pipe \ + action ct clear pipe \ + action skbedit ptype host pipe \ + action mirred ingress redirect dev $h1 + tc filter add dev $h1 protocol ip pref 101 handle 101 egress flower \ + $tcflags ip_proto icmp \ + action mirred ingress redirect dev $h1 + tc filter add dev $h1 protocol ip pref 102 handle 102 ingress flower \ + ip_proto icmp \ + action drop + + ip vrf exec v$h1 nc --recv-only -w10 -l -p 12345 -o $tmpfile1 & + local rpid=$! + ip vrf exec v$h1 nc -w1 --send-only 192.0.2.2 12345 <$tmpfile + wait -n $rpid + cmp -s $tmpfile $tmpfile1 + check_err $? "server output check failed" + + $MZ $h1 -c 10 -p 64 -a $h1mac -b $h1mac -A 192.0.2.1 -B 192.0.2.1 \ + -t icmp "ping,id=42,seq=5" -q + tc_check_packets "dev $h1 egress" 101 10 + check_err $? "didn't mirred redirect ICMP" + tc_check_packets "dev $h1 ingress" 102 10 + check_err $? "didn't drop mirred ICMP" + local overlimits=$(tc_rule_stats_get ${h1} 101 egress .overlimits) + test ${overlimits} = 10 + check_err $? "wrong overlimits, expected 10 got ${overlimits}" + + tc filter del dev $h1 egress protocol ip pref 100 handle 100 flower + tc filter del dev $h1 egress protocol ip pref 101 handle 101 flower + tc filter del dev $h1 ingress protocol ip pref 102 handle 102 flower + + rm -f $tmpfile $tmpfile1 + log_test "mirred_egress_to_ingress_tcp ($tcflags)" +} + setup_prepare() { h1=${NETIFS[p1]}