[{"id":1795456,"web_url":"http://patchwork.ozlabs.org/comment/1795456/","msgid":"<20171029.234157.1495300486874041519.davem@davemloft.net>","list_archive_url":null,"date":"2017-10-29T14:41:57","subject":"Re: [Patch net 00/16] net_sched: fix races with RCU callbacks","submitter":{"id":15,"url":"http://patchwork.ozlabs.org/api/people/15/","name":"David Miller","email":"davem@davemloft.net"},"content":"From: Cong Wang \nDate: Thu, 26 Oct 2017 18:24:27 -0700\n\n> Recently, the RCU callbacks used in TC filters and TC actions keep\n> drawing my attention, they introduce at least 4 race condition bugs:\n ...\n> As suggested by Paul, we could defer the work to a workqueue and\n> gain the permission of holding RTNL again without any performance\n> impact, however, in tcf_block_put() we could have a deadlock when\n> flushing workqueue while hodling RTNL lock, the trick here is to\n> defer the work itself in workqueue and make it queued after all\n> other works so that we keep the same ordering to avoid any\n> use-after-free. Please see the first patch for details.\n> \n> Patch 1 introduces the infrastructure, patch 2~12 move each\n> tc filter to the new tc filter workqueue, patch 13 adds\n> an assertion to catch potential bugs like this, patch 14\n> closes another rcu callback race, patch 15 and patch 16 add\n> new test cases.\n\nI know Eric has some reservations about how things have become in\nthis layer, but we have to fix this for 'net' somehow.\n\nSo I've applied this series, thanks.","headers":{"Return-Path":"","X-Original-To":"patchwork-incoming@ozlabs.org","Delivered-To":"patchwork-incoming@ozlabs.org","Authentication-Results":"ozlabs.org;\n\tspf=none (mailfrom) smtp.mailfrom=vger.kernel.org\n\t(client-ip=209.132.180.67; helo=vger.kernel.org;\n\tenvelope-from=netdev-owner@vger.kernel.org;\n\treceiver=)","Received":["from vger.kernel.org (vger.kernel.org [209.132.180.67])\n\tby ozlabs.org (Postfix) with ESMTP id 3yQ0gg6B4Nz9t2M\n\tfor ;\n\tMon, 30 Oct 2017 01:42:07 +1100 (AEDT)","(majordomo@vger.kernel.org) by vger.kernel.org via listexpand\n\tid S1751487AbdJ2OmE (ORCPT );\n\tSun, 29 Oct 2017 10:42:04 -0400","from shards.monkeyblade.net ([184.105.139.130]:50752 \"EHLO\n\tshards.monkeyblade.net\" rhost-flags-OK-OK-OK-OK) by vger.kernel.org\n\twith ESMTP id S1751269AbdJ2OmD (ORCPT\n\t); Sun, 29 Oct 2017 10:42:03 -0400","from localhost (unknown [106.252.28.196])\n\t(using TLSv1 with cipher AES256-SHA (256/256 bits))\n\t(Client did not present a certificate)\n\t(Authenticated sender: davem-davemloft)\n\tby shards.monkeyblade.net (Postfix) with ESMTPSA id 2944B12F78B86;\n\tSun, 29 Oct 2017 07:41:59 -0700 (PDT)"],"Date":"Sun, 29 Oct 2017 23:41:57 +0900 (KST)","Message-Id":"<20171029.234157.1495300486874041519.davem@davemloft.net>","To":"xiyou.wangcong@gmail.com","Cc":"netdev@vger.kernel.org, chrism@mellanox.com, daniel@iogearbox.net,\n\tjiri@resnulli.us, john.fastabend@gmail.com, jhs@mojatatu.com,\n\tpaulmck@linux.vnet.ibm.com","Subject":"Re: [Patch net 00/16] net_sched: fix races with RCU callbacks","From":"David Miller ","In-Reply-To":"<20171027012443.3306-1-xiyou.wangcong@gmail.com>","References":"<20171027012443.3306-1-xiyou.wangcong@gmail.com>","X-Mailer":"Mew version 6.7 on Emacs 25.3 / Mule 6.0 (HANACHIRUSATO)","Mime-Version":"1.0","Content-Type":"Text/Plain; charset=us-ascii","Content-Transfer-Encoding":"7bit","X-Greylist":"Sender succeeded SMTP AUTH, not delayed by\n\tmilter-greylist-4.5.12 (shards.monkeyblade.net\n\t[149.20.54.216]); Sun, 29 Oct 2017 07:42:03 -0700 (PDT)","Sender":"netdev-owner@vger.kernel.org","Precedence":"bulk","List-ID":"","X-Mailing-List":"netdev@vger.kernel.org"}},{"id":1796311,"web_url":"http://patchwork.ozlabs.org/comment/1796311/","msgid":"","list_archive_url":null,"date":"2017-10-30T22:39:03","subject":"Re: [Patch net 00/16] net_sched: fix races with RCU callbacks","submitter":{"id":70959,"url":"http://patchwork.ozlabs.org/api/people/70959/","name":"Lucas Bates","email":"lucasb@mojatatu.com"},"content":"e.On Thu, Oct 26, 2017 at 9:24 PM, Cong Wang wrote:\n> Recently, the RCU callbacks used in TC filters and TC actions keep\n> drawing my attention, they introduce at least 4 race condition bugs:\n\n> As suggested by Paul, we could defer the work to a workqueue and\n> gain the permission of holding RTNL again without any performance\n> impact, however, in tcf_block_put() we could have a deadlock when\n> flushing workqueue while hodling RTNL lock, the trick here is to\n> defer the work itself in workqueue and make it queued after all\n> other works so that we keep the same ordering to avoid any\n> use-after-free. Please see the first patch for details.\n\nCong, I don't believe the problem's been resolved just yet.... I have\na new kernel, compiled just today and I'm still tripping over a kernel\nbug in this scenario when I run Chris' new test case.\n\nI'm doing this on a machine where I don't have a spare device to use\non the run. Instead I created a veth pair that will have one end\nmigrated into the container.\n\nThe bug isn't consistent. I'm running into it anywhere between one and\nfour runs of the d052 test case.\n\nSteps to reproduce:\n\nsudo ip li add foo type veth\nsudo ./tdc.py -d foo -c flower\n[repeat until kernel bug encountered]","headers":{"Return-Path":"","X-Original-To":"patchwork-incoming@ozlabs.org","Delivered-To":"patchwork-incoming@ozlabs.org","Authentication-Results":["ozlabs.org;\n\tspf=none (mailfrom) smtp.mailfrom=vger.kernel.org\n\t(client-ip=209.132.180.67; helo=vger.kernel.org;\n\tenvelope-from=netdev-owner@vger.kernel.org;\n\treceiver=)","ozlabs.org; dkim=pass (2048-bit key;\n\tunprotected) header.d=mojatatu-com.20150623.gappssmtp.com\n\theader.i=@mojatatu-com.20150623.gappssmtp.com\n\theader.b=\"z8pEe1u/\"; dkim-atps=neutral"],"Received":["from vger.kernel.org (vger.kernel.org [209.132.180.67])\n\tby ozlabs.org (Postfix) with ESMTP id 3yQqD04vXJz9sRn\n\tfor ;\n\tTue, 31 Oct 2017 09:39:28 +1100 (AEDT)","(majordomo@vger.kernel.org) by vger.kernel.org via listexpand\n\tid S1751952AbdJ3Wj0 (ORCPT );\n\tMon, 30 Oct 2017 18:39:26 -0400","from mail-wm0-f66.google.com ([74.125.82.66]:56322 \"EHLO\n\tmail-wm0-f66.google.com\" rhost-flags-OK-OK-OK-OK) by vger.kernel.org\n\twith ESMTP id S1751402AbdJ3WjZ (ORCPT\n\t); Mon, 30 Oct 2017 18:39:25 -0400","by mail-wm0-f66.google.com with SMTP id z3so19428595wme.5\n\tfor ; Mon, 30 Oct 2017 15:39:24 -0700 (PDT)","by 10.80.203.204 with HTTP; Mon, 30 Oct 2017 15:39:03 -0700 (PDT)"],"DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/relaxed;\n\td=mojatatu-com.20150623.gappssmtp.com; s=20150623;\n\th=mime-version:in-reply-to:references:from:date:message-id:subject:to\n\t:cc; bh=WPj72umI+fgQ5OTFu9ijF+D+gfblI4E0n9MRRC9hMnA=;\n\tb=z8pEe1u/uM6Xzk20dWQGkoTD7aEQT34HsUbZZrDXenJwhvtbxQJIfBpHA7cI+RSp2b\n\tE9xC3EYMUEdwszyVJPKEfwQdZXBeX31042oFbQ1X+0o/IHKUd+RXwdQkRICBlfB0LlU0\n\tovAVqmvSx24vGnJezZed5G38FzHWS+YLayBCbmQ2oeDhYVWdyZyCMlS+AtBPRH6cp4s0\n\t+U1qmsDvSN7DPeXW4C334k5myW9DNXv74BIMjhcPJQni0q8EgyXG7BX1shFb09AgNC7R\n\tLg47KZtUKsqsVS1lEzljV/bgE+SgWv6FoNmeXh3Iqau5Jha098MVj662/S/D5wPBr95M\n\tXXfg==","X-Google-DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/relaxed;\n\td=1e100.net; s=20161025;\n\th=x-gm-message-state:mime-version:in-reply-to:references:from:date\n\t:message-id:subject:to:cc;\n\tbh=WPj72umI+fgQ5OTFu9ijF+D+gfblI4E0n9MRRC9hMnA=;\n\tb=tfyGj7n4n/fpvmnwznRjxL1RawQfOphUTIIYGINPIjroLx+VCJMZRmlheMZ9HhFawN\n\tIYxtM8odGYgsiY5qKJXiJ9JjhOtZeBJFcqN0E9IpCfgNlE33AhpQ2uzPPHqTVYuI9ZNd\n\tv1M+kuMpntSZHB8kA7mM4EVMQtGET+ASBffqHmRHA3nMS9dFa82eE6KCp34X8LojQEOM\n\tCXmEgUYm3JrmNGkubwepUyZ5KyLwfmnwY9hGgtrWBKUUJZZ5y3xwAhTXLEbqDjecPgsd\n\tLmCSDaZNlDM42j/cIAqqiD1mrfnR0a/BEE8rUzXn916/kfbLapBvddRyo9k0FoIVftWq\n\ttuZA==","X-Gm-Message-State":"AMCzsaWzkyjByEwke1B+eL+NuiaITzaBwe70b3BA9wrCt+z9aY8rIJxD\n\tEmbqKqF6RnHBctSnxQWjW9TxCu7F5+AeBhIBgbbuGQ==","X-Google-Smtp-Source":"ABhQp+TT1kkBI4FWdI8pyy6yFsgzWNoWPIaSLGw4PZ5xWLyUtBk50cVibHJ6AlcRZpajOS/HsmoGOR8wX5+f0Mrqmn4=","X-Received":"by 10.80.194.217 with SMTP id u25mr312173edf.260.1509403164309; \n\tMon, 30 Oct 2017 15:39:24 -0700 (PDT)","MIME-Version":"1.0","In-Reply-To":"<20171027012443.3306-1-xiyou.wangcong@gmail.com>","References":"<20171027012443.3306-1-xiyou.wangcong@gmail.com>","From":"Lucas Bates ","Date":"Mon, 30 Oct 2017 18:39:03 -0400","Message-ID":"","Subject":"Re: [Patch net 00/16] net_sched: fix races with RCU callbacks","To":"Cong Wang ","Cc":"netdev@vger.kernel.org, Chris Mi ,\n\tDaniel Borkmann , Jiri Pirko ,\n\tJohn Fastabend ,\n\tJamal Hadi Salim ,\n\t\"Paul E. McKenney\" ","Content-Type":"text/plain; charset=\"UTF-8\"","Sender":"netdev-owner@vger.kernel.org","Precedence":"bulk","List-ID":"","X-Mailing-List":"netdev@vger.kernel.org"}},{"id":1796329,"web_url":"http://patchwork.ozlabs.org/comment/1796329/","msgid":"","list_archive_url":null,"date":"2017-10-30T23:12:39","subject":"Re: [Patch net 00/16] net_sched: fix races with RCU callbacks","submitter":{"id":211,"url":"http://patchwork.ozlabs.org/api/people/211/","name":"Cong Wang","email":"xiyou.wangcong@gmail.com"},"content":"On Mon, Oct 30, 2017 at 3:39 PM, Lucas Bates wrote:\n> e.On Thu, Oct 26, 2017 at 9:24 PM, Cong Wang wrote:\n>> Recently, the RCU callbacks used in TC filters and TC actions keep\n>> drawing my attention, they introduce at least 4 race condition bugs:\n> \n>> As suggested by Paul, we could defer the work to a workqueue and\n>> gain the permission of holding RTNL again without any performance\n>> impact, however, in tcf_block_put() we could have a deadlock when\n>> flushing workqueue while hodling RTNL lock, the trick here is to\n>> defer the work itself in workqueue and make it queued after all\n>> other works so that we keep the same ordering to avoid any\n>> use-after-free. Please see the first patch for details.\n>\n> Cong, I don't believe the problem's been resolved just yet.... I have\n> a new kernel, compiled just today and I'm still tripping over a kernel\n> bug in this scenario when I run Chris' new test case.\n>\n\nWithout a stack trace, I can't do anything. \"a kernel bug\" could\nbe anything, why do you believe it is caused by this patchset?\n\nNote, there is a use-after-free bug caused by this patchset, the\nfix is already on netdev in case you hit the same bug.\n\n> I'm doing this on a machine where I don't have a spare device to use\n> on the run. Instead I created a veth pair that will have one end\n> migrated into the container.\n>\n> The bug isn't consistent. I'm running into it anywhere between one and\n> four runs of the d052 test case.\n>\n> Steps to reproduce:\n>\n> sudo ip li add foo type veth\n> sudo ./tdc.py -d foo -c flower\n> [repeat until kernel bug encountered]\n\nI tested with basic and u32 filter, since it is not specific to any type\nof filters, all filters had the same problem. I will try flower filter too\nbefore you provide a stack trace or a detailed report.","headers":{"Return-Path":"","X-Original-To":"patchwork-incoming@ozlabs.org","Delivered-To":"patchwork-incoming@ozlabs.org","Authentication-Results":["ozlabs.org;\n\tspf=none (mailfrom) smtp.mailfrom=vger.kernel.org\n\t(client-ip=209.132.180.67; helo=vger.kernel.org;\n\tenvelope-from=netdev-owner@vger.kernel.org;\n\treceiver=)","ozlabs.org; dkim=pass (2048-bit key;\n\tunprotected) header.d=gmail.com header.i=@gmail.com\n\theader.b=\"ncg2R53x\"; dkim-atps=neutral"],"Received":["from vger.kernel.org (vger.kernel.org [209.132.180.67])\n\tby ozlabs.org (Postfix) with ESMTP id 3yQqyn6pzHz9t3m\n\tfor ;\n\tTue, 31 Oct 2017 10:13:05 +1100 (AEDT)","(majordomo@vger.kernel.org) by vger.kernel.org via listexpand\n\tid S1753315AbdJ3XNB (ORCPT );\n\tMon, 30 Oct 2017 19:13:01 -0400","from mail-pf0-f195.google.com ([209.85.192.195]:51444 \"EHLO\n\tmail-pf0-f195.google.com\" rhost-flags-OK-OK-OK-OK) by vger.kernel.org\n\twith ESMTP id S1753131AbdJ3XNB (ORCPT\n\t); Mon, 30 Oct 2017 19:13:01 -0400","by mail-pf0-f195.google.com with SMTP id n14so12218401pfh.8\n\tfor ; Mon, 30 Oct 2017 16:13:00 -0700 (PDT)","by 10.100.170.205 with HTTP; Mon, 30 Oct 2017 16:12:39 -0700 (PDT)"],"DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/relaxed;\n\td=gmail.com; s=20161025;\n\th=mime-version:in-reply-to:references:from:date:message-id:subject:to\n\t:cc; bh=tYBwSaQzf9X40Y1vs/xXjTX9WZmc8hQz4GEHh5TL5lU=;\n\tb=ncg2R53xDonhG22M89ZtIDjTAEk3jeFwcg58Cii/EwuJqIpmBgzE9v3q/bmtG9dQyT\n\tIZVEU971OkQlyrod3+V6tUAbZxT+GHwWms8l8f1CeCVE2Jpr7ati4zmy/pwKIk0FUjOq\n\tlnPK3L/HNP4Tn0Afp4/EC+KSbd0PYjsLyPiqk1c8cnYc1OJTB6+0P7qCHDFjPn3QxDN0\n\tV/M2n5CZQY3UJefbCSd3bcgezubP7FKGfWJX7Spo9QKE5tNs45a3CE0I4EL03Gpu/hhS\n\t+rnF+GRNXBZnWtSZzFb8IQYvkfPxU0fV4UgGL2ksf+YDfNrxxDvjdobs1qsmeBop3ROP\n\tUlCA==","X-Google-DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/relaxed;\n\td=1e100.net; s=20161025;\n\th=x-gm-message-state:mime-version:in-reply-to:references:from:date\n\t:message-id:subject:to:cc;\n\tbh=tYBwSaQzf9X40Y1vs/xXjTX9WZmc8hQz4GEHh5TL5lU=;\n\tb=EW6AozvHyLgFh2MXZPDoqaIrk6zoRs8vIGMyrHNQ8ZNspUxRiFYs84yStP1qHZjuS9\n\t+ojZtKjxbLfNt2xJGgswvuQB7vjP/PthjmzutHi3hXoyy86ch2Iztip+ALfEmfQiVU4V\n\tJ8RfuA1AWcjinoSZrxQjJ42hNJDT2LsAfqSRR2JpwP5U0GaTcb1v0yOBsbY2VM3xURnA\n\tVuGu5iuXQFqOgALssPLr0okYx4o06yOVB5SQnImHBfl3GWfVF3qfxj8uusYvu1e7l0B1\n\twSqRsUCZjxNjUVem0GP2WbxX3IqYgeoqR6HF853MdilbmqvT1sndh5nWmTQJ3XZHLTEm\n\t5tfA==","X-Gm-Message-State":"AMCzsaXxBH05NtG0aMKvnKgfsaNQs5sXhEMygpeKOPn1g/JrJnLM6Y0x\n\tiIXLvYcBEXuh2q5bGdpI5XiobDedzRqYbIfZdIg=","X-Google-Smtp-Source":"ABhQp+SrM9KdFedTtZJnlU2la536bMOtlleE3g+wF9F7WOPFAQ4f8PiTo7IcazdHP4TJSpMDMugWG7y1aHGNKm4REk8=","X-Received":"by 10.101.85.4 with SMTP id f4mr15385pgr.10.1509405180442;\n\tMon, 30 Oct 2017 16:13:00 -0700 (PDT)","MIME-Version":"1.0","In-Reply-To":"","References":"<20171027012443.3306-1-xiyou.wangcong@gmail.com>\n\t","From":"Cong Wang ","Date":"Mon, 30 Oct 2017 16:12:39 -0700","Message-ID":"","Subject":"Re: [Patch net 00/16] net_sched: fix races with RCU callbacks","To":"Lucas Bates ","Cc":"Linux Kernel Network Developers ,\n\tChris Mi , Daniel Borkmann ,\n\tJiri Pirko , John Fastabend ,\n\tJamal Hadi Salim ,\n\t\"Paul E. McKenney\" ","Content-Type":"text/plain; charset=\"UTF-8\"","Sender":"netdev-owner@vger.kernel.org","Precedence":"bulk","List-ID":"","X-Mailing-List":"netdev@vger.kernel.org"}},{"id":1796367,"web_url":"http://patchwork.ozlabs.org/comment/1796367/","msgid":"","list_archive_url":null,"date":"2017-10-31T01:46:16","subject":"Re: [Patch net 00/16] net_sched: fix races with RCU callbacks","submitter":{"id":70959,"url":"http://patchwork.ozlabs.org/api/people/70959/","name":"Lucas Bates","email":"lucasb@mojatatu.com"},"content":"On Mon, Oct 30, 2017 at 7:12 PM, Cong Wang wrote:\n> On Mon, Oct 30, 2017 at 3:39 PM, Lucas Bates wrote:\n>> e.On Thu, Oct 26, 2017 at 9:24 PM, Cong Wang wrote:\n>>> Recently, the RCU callbacks used in TC filters and TC actions keep\n>>> drawing my attention, they introduce at least 4 race condition bugs:\n>> \n>>> As suggested by Paul, we could defer the work to a workqueue and\n>>> gain the permission of holding RTNL again without any performance\n>>> impact, however, in tcf_block_put() we could have a deadlock when\n>>> flushing workqueue while hodling RTNL lock, the trick here is to\n>>> defer the work itself in workqueue and make it queued after all\n>>> other works so that we keep the same ordering to avoid any\n>>> use-after-free. Please see the first patch for details.\n>>\n>> Cong, I don't believe the problem's been resolved just yet.... I have\n>> a new kernel, compiled just today and I'm still tripping over a kernel\n>> bug in this scenario when I run Chris' new test case.\n>>\n>\n> Without a stack trace, I can't do anything. \"a kernel bug\" could\n> be anything, why do you believe it is caused by this patchset?\n>\n(Sending again due to email format issues)\n\nThe stack trace I saw touched on the same code that was affected by\nthe patchset. I've sent a photo of the trace to you directly - I'm\nsorry, I should have sent it earlier. I also apologize for having to\nsend the photo but I was doing the testing on a small device lacking\nany kind of serial console access.\n\n\n>> I'm doing this on a machine where I don't have a spare device to use\n>> on the run. Instead I created a veth pair that will have one end\n>> migrated into the container.\n>>\n>> The bug isn't consistent. I'm running into it anywhere between one and\n>> four runs of the d052 test case.\n>>\n>> Steps to reproduce:\n>>\n>> sudo ip li add foo type veth\n>> sudo ./tdc.py -d foo -c flower\n>> [repeat until kernel bug encountered]\n>\n> I tested with basic and u32 filter, since it is not specific to any type\n> of filters, all filters had the same problem. I will try flower filter too\n> before you provide a stack trace or a detailed report.\nIf you need more information or need me to try something else, I'll be\nable to tomorrow.","headers":{"Return-Path":"","X-Original-To":"patchwork-incoming@ozlabs.org","Delivered-To":"patchwork-incoming@ozlabs.org","Authentication-Results":["ozlabs.org;\n\tspf=none (mailfrom) smtp.mailfrom=vger.kernel.org\n\t(client-ip=209.132.180.67; helo=vger.kernel.org;\n\tenvelope-from=netdev-owner@vger.kernel.org;\n\treceiver=)","ozlabs.org; dkim=pass (2048-bit key;\n\tunprotected) header.d=mojatatu-com.20150623.gappssmtp.com\n\theader.i=@mojatatu-com.20150623.gappssmtp.com\n\theader.b=\"EpWh4yNI\"; dkim-atps=neutral"],"Received":["from vger.kernel.org (vger.kernel.org [209.132.180.67])\n\tby ozlabs.org (Postfix) with ESMTP id 3yQvN444JNz9t2r\n\tfor ;\n\tTue, 31 Oct 2017 12:46:44 +1100 (AEDT)","(majordomo@vger.kernel.org) by vger.kernel.org via listexpand\n\tid S1752905AbdJaBqj (ORCPT );\n\tMon, 30 Oct 2017 21:46:39 -0400","from mail-wm0-f68.google.com ([74.125.82.68]:46968 \"EHLO\n\tmail-wm0-f68.google.com\" rhost-flags-OK-OK-OK-OK) by vger.kernel.org\n\twith ESMTP id S1752368AbdJaBqi (ORCPT\n\t); Mon, 30 Oct 2017 21:46:38 -0400","by mail-wm0-f68.google.com with SMTP id m72so19388555wmc.1\n\tfor ; Mon, 30 Oct 2017 18:46:37 -0700 (PDT)","by 10.80.203.204 with HTTP; Mon, 30 Oct 2017 18:46:16 -0700 (PDT)"],"DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/relaxed;\n\td=mojatatu-com.20150623.gappssmtp.com; s=20150623;\n\th=mime-version:in-reply-to:references:from:date:message-id:subject:to;\n\tbh=GzL/FnBuypiQP+Wl1XHs7lMDWpaUpPwvfEN3wZum49s=;\n\tb=EpWh4yNIex6S9N6ge+R0ELBFxxHfKQciBFBUIpzV2QSijJw1HVDXKH+ROneGCETDgZ\n\tu8PMD3g/S4H7HxpMsZ3YH2eacwzHrYFOzJhm9bHbPdd/DS7aZwkLrSSP7erZsC5F9t7u\n\td3ywNw/+obl4ABEAuUuzp9ygIHxgMg0hbTlFuP6d+02eWr3+ip1+PciMyhax2hl/8HSA\n\tN63AK29wqn9UU/mqH9sKpEBk5f/Fhp4rG1lL7+HHbHv1eJiCpTP0MKuJBfe9nG1BCC1D\n\tVdYF7Iq4igYuF7w/lutGxfEtx8KDzkxd4fS/FUK/o2cmYXwlDP9ld44OMwulEXQX1TCp\n\tCr3g==","X-Google-DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/relaxed;\n\td=1e100.net; s=20161025;\n\th=x-gm-message-state:mime-version:in-reply-to:references:from:date\n\t:message-id:subject:to;\n\tbh=GzL/FnBuypiQP+Wl1XHs7lMDWpaUpPwvfEN3wZum49s=;\n\tb=nd20C/16PZm0O9jMq2Plf+t+T7OSAPHhpEbBHiJ2xH61uoBGxCcUWh7PCtiHljnf4T\n\tlzD2Q4Nj2GCGbs0k31hehIJqkyA8qvOTbyWwTOdGAtdg0ccNbndlLSehDZD0qz7OlaGL\n\tLT1Q7YMMkTGSV1t+LO6X1i3ZJzhX3RuFu3IwbgeIaRVsXlGC9PC3oLm9olSPhY9VYnDi\n\t5iq/7O8Brid+vyF0Zq48Ed19Sjn/B8y+rAB/iUYWsdpr8SWOJ69II1NeLN98v7EbFtLP\n\tcYpVM849iuCdaUfMvBFc4s6Jp5K6aWNSZGBY9JUVuaOwjlUcUzjP4/R3ggXrZmNW7DCm\n\tI/jA==","X-Gm-Message-State":"AMCzsaUauE9aK9VqMqvZ/j7jyiltVDRsI1zaA2GNveSdqK2f2t8EMvef\n\t8oN45MTb7t/4uP3cd09p6sPXjg6373wp9/1bUXkrKQ==","X-Google-Smtp-Source":"ABhQp+QgQlZcwJRSv+sTtOUuOSeJSe9BituM1ZuUpZdkvlspwXP5qNqwYRo8p/3/qLrbyf+QBwlhy4T9mhWi6ClC2+Q=","X-Received":"by 10.80.157.141 with SMTP id w13mr826507ede.151.1509414397160; \n\tMon, 30 Oct 2017 18:46:37 -0700 (PDT)","MIME-Version":"1.0","In-Reply-To":"","References":"<20171027012443.3306-1-xiyou.wangcong@gmail.com>\n\t\n\t","From":"Lucas Bates ","Date":"Mon, 30 Oct 2017 21:46:16 -0400","Message-ID":"","Subject":"Re: [Patch net 00/16] net_sched: fix races with RCU callbacks","To":"Linux Kernel Network Developers ","Content-Type":"text/plain; charset=\"UTF-8\"","Sender":"netdev-owner@vger.kernel.org","Precedence":"bulk","List-ID":"","X-Mailing-List":"netdev@vger.kernel.org"}},{"id":1796420,"web_url":"http://patchwork.ozlabs.org/comment/1796420/","msgid":"","list_archive_url":null,"date":"2017-10-31T05:44:50","subject":"Re: [Patch net 00/16] net_sched: fix races with RCU callbacks","submitter":{"id":211,"url":"http://patchwork.ozlabs.org/api/people/211/","name":"Cong Wang","email":"xiyou.wangcong@gmail.com"},"content":"On Mon, Oct 30, 2017 at 6:03 PM, Lucas Bates wrote:\n> On Oct 30, 2017 19:13, \"Cong Wang\" wrote:\n>>\n>> On Mon, Oct 30, 2017 at 3:39 PM, Lucas Bates wrote:\n>> > e.On Thu, Oct 26, 2017 at 9:24 PM, Cong Wang \n>> > wrote:\n>> >> Recently, the RCU callbacks used in TC filters and TC actions keep\n>> >> drawing my attention, they introduce at least 4 race condition bugs:\n>> > \n>> >> As suggested by Paul, we could defer the work to a workqueue and\n>> >> gain the permission of holding RTNL again without any performance\n>> >> impact, however, in tcf_block_put() we could have a deadlock when\n>> >> flushing workqueue while hodling RTNL lock, the trick here is to\n>> >> defer the work itself in workqueue and make it queued after all\n>> >> other works so that we keep the same ordering to avoid any\n>> >> use-after-free. Please see the first patch for details.\n>> >\n>> > Cong, I don't believe the problem's been resolved just yet.... I have\n>> > a new kernel, compiled just today and I'm still tripping over a kernel\n>> > bug in this scenario when I run Chris' new test case.\n>> >\n>>\n>> Without a stack trace, I can't do anything. \"a kernel bug\" could\n>> be anything, why do you believe it is caused by this patchset?\n>>\n> The stack trace I saw touched on the same code that was affected by the\n> patchset. I've attached a photo of the trace - sorry, I should have sent it\n> earlier. I also apologize for having to send a photo but I was doing the\n> testing on a small device lacking any kind of serial console access.\n\n\nCan you try this patch? From your stack trace it is not clear where\nthe cause is, but we know that the crash is in __tcf_idr_release(),\nthis is how I came up with the following patch:\n\n\ndiff --git a/include/net/act_api.h b/include/net/act_api.h\nindex b944e0eb93be..5072446d5f06 100644\n--- a/include/net/act_api.h\n+++ b/include/net/act_api.h\n@@ -122,7 +122,9 @@ void tcf_idrinfo_destroy(const struct tc_action_ops *ops,\n\n static inline void tc_action_net_exit(struct tc_action_net *tn)\n {\n+ rtnl_lock();\n tcf_idrinfo_destroy(tn->ops, tn->idrinfo);\n+ rtnl_unlock();\n kfree(tn->idrinfo);\n }\n\n\n\n>\n> If you need more information or need me to try something else, I'll be able\n> to tomorrow.\n>\n\nI will look deeper tomorrow. It doesn't look like caused by this patchset\nso far, probably yet another missing rtnl like what the above patch shows.\n\n\nThanks!","headers":{"Return-Path":"","X-Original-To":"patchwork-incoming@ozlabs.org","Delivered-To":"patchwork-incoming@ozlabs.org","Authentication-Results":["ozlabs.org;\n\tspf=none (mailfrom) smtp.mailfrom=vger.kernel.org\n\t(client-ip=209.132.180.67; helo=vger.kernel.org;\n\tenvelope-from=netdev-owner@vger.kernel.org;\n\treceiver=)","ozlabs.org; dkim=pass (2048-bit key;\n\tunprotected) header.d=gmail.com header.i=@gmail.com\n\theader.b=\"ZK+Pb3ce\"; dkim-atps=neutral"],"Received":["from vger.kernel.org (vger.kernel.org [209.132.180.67])\n\tby ozlabs.org (Postfix) with ESMTP id 3yR0gG6g7Pz9s7g\n\tfor ;\n\tTue, 31 Oct 2017 16:45:14 +1100 (AEDT)","(majordomo@vger.kernel.org) by vger.kernel.org via listexpand\n\tid S1753359AbdJaFpM (ORCPT );\n\tTue, 31 Oct 2017 01:45:12 -0400","from mail-pg0-f65.google.com ([74.125.83.65]:51311 \"EHLO\n\tmail-pg0-f65.google.com\" rhost-flags-OK-OK-OK-OK) by vger.kernel.org\n\twith ESMTP id S1753229AbdJaFpL (ORCPT\n\t); Tue, 31 Oct 2017 01:45:11 -0400","by mail-pg0-f65.google.com with SMTP id p9so13733391pgc.8\n\tfor ; Mon, 30 Oct 2017 22:45:11 -0700 (PDT)","by 10.100.170.205 with HTTP; Mon, 30 Oct 2017 22:44:50 -0700 (PDT)"],"DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/relaxed;\n\td=gmail.com; s=20161025;\n\th=mime-version:in-reply-to:references:from:date:message-id:subject:to\n\t:cc; bh=jB3lTIJc9BWcik9an5tx4FjoBZicWjgkVpSLaF6MxCM=;\n\tb=ZK+Pb3cenSVa0SO/KQUvKCACQCBO9/ndJWOe6JlxuzgCXQo04IHDU5bjMTWPyApyml\n\tvjdrExrwUSkTnWEZPFC2Rm+WsDRf3M56plIMXQa7blwlPre/4k3tK11eDBZFfBpr65lQ\n\t547NgsTRUNrFs25GGu1LchHaAqdblyyGbZFoHhuWrMuIclhoWAEJ3Zok/7rxboDWU1SF\n\tXKgz1QrVc8KSDXflQ2JUfP7j50dNMFYEfBhi1mTkK9LHIXO3LlN3J4vSYSVG1jg9XgfG\n\tGhsDZRqrZLLDo999hZ0q0UUWQ+l1XwhUW8s0rC2fP8wtUKHS/TmHZ84yRrFDeuujgSHk\n\tPVKw==","X-Google-DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/relaxed;\n\td=1e100.net; s=20161025;\n\th=x-gm-message-state:mime-version:in-reply-to:references:from:date\n\t:message-id:subject:to:cc;\n\tbh=jB3lTIJc9BWcik9an5tx4FjoBZicWjgkVpSLaF6MxCM=;\n\tb=a38sjhT6RcRjjiMRCQC4Qx9qpiVvMzP1AWpF8tY66TiqA7DAnJ6xrDZF6qLFeII8Dv\n\tZ9CRtZNKdqmigODLtd4Mz3BQ8qNJk5H8GfWjpP418mPKmr8Ssj4FrJamjKwrFvdPm8Tb\n\tPHtcokRjrso5OnU4y5vSiQJGFeRK+6B7mdy5R+dRUaphNzJWDifw05wHmVAPyyOh05eS\n\tC4dEo0Jz2ZLqQXteWchb+f7pzaxG0z/+cTEldec+Dhgce7+vhJO/MkFm0FKndbEFRNRi\n\tgUkJY2q98YYobc+P+cIi69SbkF4LjKENfYO8JkFy0f5yOSHbixckWxK1/nQC+ML5NLkC\n\t1z8g==","X-Gm-Message-State":"AMCzsaXGWlBvfrR/QxddDIAuQm5ed8BWaQrQfcIVhAZW4SFYYU0RloB2\n\tsFhcCXpmNbKA4PgWgEE4pxk8vEEMQvyKejIZFbk=","X-Google-Smtp-Source":"ABhQp+RLdZCplrRuPt+eIcRjBpH5PN6WBeWirjZQG6lkbN/Y3jO7OvQiS/QUtgZK44pPCf8ZZ+UI3LrRXx3+Yqr/BMk=","X-Received":"by 10.99.119.9 with SMTP id s9mr838244pgc.54.1509428710692; Mon,\n\t30 Oct 2017 22:45:10 -0700 (PDT)","MIME-Version":"1.0","In-Reply-To":"","References":"<20171027012443.3306-1-xiyou.wangcong@gmail.com>\n\t\n\t\n\t","From":"Cong Wang ","Date":"Mon, 30 Oct 2017 22:44:50 -0700","Message-ID":"","Subject":"Re: [Patch net 00/16] net_sched: fix races with RCU callbacks","To":"Lucas Bates ","Cc":"Linux Kernel Network Developers ,\n\tChris Mi , Daniel Borkmann ,\n\tJiri Pirko , John Fastabend ,\n\tJamal Hadi Salim ,\n\t\"Paul E. McKenney\" ","Content-Type":"text/plain; charset=\"UTF-8\"","Sender":"netdev-owner@vger.kernel.org","Precedence":"bulk","List-ID":"","X-Mailing-List":"netdev@vger.kernel.org"}},{"id":1796573,"web_url":"http://patchwork.ozlabs.org/comment/1796573/","msgid":"<0dfff609-d67e-0701-399f-1dbe2ef09bf3@mojatatu.com>","list_archive_url":null,"date":"2017-10-31T11:00:22","subject":"Re: [Patch net 00/16] net_sched: fix races with RCU callbacks","submitter":{"id":9380,"url":"http://patchwork.ozlabs.org/api/people/9380/","name":"Jamal Hadi Salim","email":"jhs@mojatatu.com"},"content":"On 17-10-31 01:44 AM, Cong Wang wrote:\n> On Mon, Oct 30, 2017 at 6:03 PM, Lucas Bates wrote:\n>> On Oct 30, 2017 19:13, \"Cong Wang\" wrote:\n>>>\n\n> Can you try this patch? From your stack trace it is not clear where\n> the cause is, but we know that the crash is in __tcf_idr_release(),\n> this is how I came up with the following patch:\n> \n> \n> diff --git a/include/net/act_api.h b/include/net/act_api.h\n> index b944e0eb93be..5072446d5f06 100644\n> --- a/include/net/act_api.h\n> +++ b/include/net/act_api.h\n> @@ -122,7 +122,9 @@ void tcf_idrinfo_destroy(const struct tc_action_ops *ops,\n> \n> static inline void tc_action_net_exit(struct tc_action_net *tn)\n> {\n> + rtnl_lock();\n> tcf_idrinfo_destroy(tn->ops, tn->idrinfo);\n> + rtnl_unlock();\n> kfree(tn->idrinfo);\n> }\n> \n>\n\nLooks like an excellent bet;-> tdc kills the container at the end of the\ntest.\n\ncheers,\njamal","headers":{"Return-Path":"","X-Original-To":"patchwork-incoming@ozlabs.org","Delivered-To":"patchwork-incoming@ozlabs.org","Authentication-Results":["ozlabs.org;\n\tspf=none (mailfrom) smtp.mailfrom=vger.kernel.org\n\t(client-ip=209.132.180.67; helo=vger.kernel.org;\n\tenvelope-from=netdev-owner@vger.kernel.org;\n\treceiver=)","ozlabs.org; dkim=pass (2048-bit key;\n\tunprotected) header.d=mojatatu-com.20150623.gappssmtp.com\n\theader.i=@mojatatu-com.20150623.gappssmtp.com\n\theader.b=\"yQ3+heJT\"; dkim-atps=neutral"],"Received":["from vger.kernel.org (vger.kernel.org [209.132.180.67])\n\tby ozlabs.org (Postfix) with ESMTP id 3yR7g12CxYz9sRg\n\tfor ;\n\tTue, 31 Oct 2017 22:00:29 +1100 (AEDT)","(majordomo@vger.kernel.org) by vger.kernel.org via listexpand\n\tid S1752550AbdJaLA0 (ORCPT );\n\tTue, 31 Oct 2017 07:00:26 -0400","from mail-io0-f195.google.com ([209.85.223.195]:55180 \"EHLO\n\tmail-io0-f195.google.com\" rhost-flags-OK-OK-OK-OK) by vger.kernel.org\n\twith ESMTP id S1751667AbdJaLAZ (ORCPT\n\t); Tue, 31 Oct 2017 07:00:25 -0400","by mail-io0-f195.google.com with SMTP id e89so34040174ioi.11\n\tfor ; Tue, 31 Oct 2017 04:00:24 -0700 (PDT)","from [10.0.0.13] (135-23-93-187.cpe.pppoe.ca. [135.23.93.187])\n\tby smtp.googlemail.com with ESMTPSA id\n\tr79sm747368ita.0.2017.10.31.04.00.23\n\t(version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128);\n\tTue, 31 Oct 2017 04:00:23 -0700 (PDT)"],"DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/relaxed;\n\td=mojatatu-com.20150623.gappssmtp.com; s=20150623;\n\th=subject:to:cc:references:from:message-id:date:user-agent\n\t:mime-version:in-reply-to:content-language:content-transfer-encoding; \n\tbh=RrngNzwD+sbjHgFGeqZF5JelBBsgnv+UDm2Y9daNWw0=;\n\tb=yQ3+heJTKSZHHppcDR7wKawpVxYFeZKrfVnCcvSYfgduv9PJL2TGYCM+ofAG65vNa5\n\tBetzYQzwdL1FCIHFggemNu68V2suXFjPlwk4R+0bmkW9TYxUVHYbWFxYGq8D422WeqVM\n\t9EqZbMRrNuOU8VnvWI8lVRJlUn1zjKPu5w3mFiALanA0nxbP4P0n1bGVcjIV+n5LiSE8\n\tjyORNmDmzjJui8aWXN4hzA7ykGNlaQOW2bshyQCOLi592uETfpIX/ZLZNufTvXEJYhZY\n\t6DqNc+2N6NDTuzKwjj3nc530glzeDzgy2eM2avW4bfpjnz2Fi/hCaDJ26zPbRJQIMxgF\n\ta32g==","X-Google-DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/relaxed;\n\td=1e100.net; s=20161025;\n\th=x-gm-message-state:subject:to:cc:references:from:message-id:date\n\t:user-agent:mime-version:in-reply-to:content-language\n\t:content-transfer-encoding;\n\tbh=RrngNzwD+sbjHgFGeqZF5JelBBsgnv+UDm2Y9daNWw0=;\n\tb=W6R4vxpE6v8KMpLkCNyNpSCIzyGyaNubCL5YOLmLYGmUw83o0IsHHr3kua/Sn5iiWo\n\tWizxQfk8mMamlfa2XYZdQGKzXmxjBPfZyGXPqOqSEOfr1EkZLTmI9qUki+r/eGE2FeQu\n\tbCJBLuEpYfdNyMSHqz1ejKXCvM59Y4j7WAUbcrINY7RBm3DQaneXmZHENN8km4Zte8NH\n\t1S++IGqtXdFIntmBL96dutkwyaRpprznr/62YkKWbxJnuyZCz2QcjWo4iWesjg3P5hSJ\n\t20Kd3PX23JQnTRGDavj7vJq6HQwZe28vzbwd47H7JS0Ood2FtR7aQ1og3VOVY0Kog8UL\n\tOKoQ==","X-Gm-Message-State":"AMCzsaWZdZM2gqdAT1E0dEkKKZzwcolFrHTtLWrQNgf2Jy0SWivxXd3a\n\tpcHwEy7URAbXBkQl3p32PLpOLg==","X-Google-Smtp-Source":"ABhQp+RVqJ/du/5GAbfOpNNxh3SxDqmjveNhsPFPnUEB8u27UKGtNkaFYz5gsiNEJsLaGj3/5iNS/Q==","X-Received":"by 10.107.138.204 with SMTP id c73mr1776662ioj.91.1509447624505; \n\tTue, 31 Oct 2017 04:00:24 -0700 (PDT)","Subject":"Re: [Patch net 00/16] net_sched: fix races with RCU callbacks","To":"Cong Wang , Lucas Bates ","Cc":"Linux Kernel Network Developers ,\n\tChris Mi , Daniel Borkmann ,\n\tJiri Pirko , John Fastabend ,\n\t\"Paul E. McKenney\" ","References":"<20171027012443.3306-1-xiyou.wangcong@gmail.com>\n\t\n\t\n\t\n\t","From":"Jamal Hadi Salim ","Message-ID":"<0dfff609-d67e-0701-399f-1dbe2ef09bf3@mojatatu.com>","Date":"Tue, 31 Oct 2017 07:00:22 -0400","User-Agent":"Mozilla/5.0 (X11; Linux x86_64; rv:52.0) Gecko/20100101\n\tThunderbird/52.4.0","MIME-Version":"1.0","In-Reply-To":"","Content-Type":"text/plain; charset=utf-8; format=flowed","Content-Language":"en-GB","Content-Transfer-Encoding":"7bit","Sender":"netdev-owner@vger.kernel.org","Precedence":"bulk","List-ID":"","X-Mailing-List":"netdev@vger.kernel.org"}},{"id":1796871,"web_url":"http://patchwork.ozlabs.org/comment/1796871/","msgid":"","list_archive_url":null,"date":"2017-10-31T18:55:40","subject":"Re: [Patch net 00/16] net_sched: fix races with RCU callbacks","submitter":{"id":70959,"url":"http://patchwork.ozlabs.org/api/people/70959/","name":"Lucas Bates","email":"lucasb@mojatatu.com"},"content":"On Tue, Oct 31, 2017 at 7:00 AM, Jamal Hadi Salim wrote:\n> On 17-10-31 01:44 AM, Cong Wang wrote:\n>>\n>> Can you try this patch? From your stack trace it is not clear where\n>> the cause is, but we know that the crash is in __tcf_idr_release(),\n>> this is how I came up with the following patch:\n>>\n>>\n>> diff --git a/include/net/act_api.h b/include/net/act_api.h\n>> index b944e0eb93be..5072446d5f06 100644\n>> --- a/include/net/act_api.h\n>> +++ b/include/net/act_api.h\n>> @@ -122,7 +122,9 @@ void tcf_idrinfo_destroy(const struct tc_action_ops\n>> *ops,\n>>\n>> static inline void tc_action_net_exit(struct tc_action_net *tn)\n>> {\n>> + rtnl_lock();\n>> tcf_idrinfo_destroy(tn->ops, tn->idrinfo);\n>> + rtnl_unlock();\n>> kfree(tn->idrinfo);\n>> }\n>>\n>>\n>\n> Looks like an excellent bet;-> tdc kills the container at the end of the\n> test.\n\nUnfortunately it doesn't seem to have had any effect, I'm still seeing\nthe same bug as yesterday. At Jamal's suggestion I put in a delay\nafter tdc completed running the tests but before it deleted the\ncontainer - and I didn't run into the bug after many runs. I had no\nluck getting serial console access on any of our other systems so I\nhave another stack trace photo which I'll send to you directly.","headers":{"Return-Path":"","X-Original-To":"patchwork-incoming@ozlabs.org","Delivered-To":"patchwork-incoming@ozlabs.org","Authentication-Results":["ozlabs.org;\n\tspf=none (mailfrom) smtp.mailfrom=vger.kernel.org\n\t(client-ip=209.132.180.67; helo=vger.kernel.org;\n\tenvelope-from=netdev-owner@vger.kernel.org;\n\treceiver=)","ozlabs.org; dkim=pass (2048-bit key;\n\tunprotected) header.d=mojatatu-com.20150623.gappssmtp.com\n\theader.i=@mojatatu-com.20150623.gappssmtp.com\n\theader.b=\"sJGCGqE3\"; dkim-atps=neutral"],"Received":["from vger.kernel.org (vger.kernel.org [209.132.180.67])\n\tby ozlabs.org (Postfix) with ESMTP id 3yRLCn1kc1z9s7f\n\tfor ;\n\tWed, 1 Nov 2017 05:56:05 +1100 (AEDT)","(majordomo@vger.kernel.org) by vger.kernel.org via listexpand\n\tid S932701AbdJaS4C (ORCPT );\n\tTue, 31 Oct 2017 14:56:02 -0400","from mail-wm0-f52.google.com ([74.125.82.52]:51780 \"EHLO\n\tmail-wm0-f52.google.com\" rhost-flags-OK-OK-OK-OK) by vger.kernel.org\n\twith ESMTP id S932557AbdJaS4B (ORCPT\n\t); Tue, 31 Oct 2017 14:56:01 -0400","by mail-wm0-f52.google.com with SMTP id b9so854849wmh.0\n\tfor ; Tue, 31 Oct 2017 11:56:01 -0700 (PDT)","by 10.80.203.204 with HTTP; Tue, 31 Oct 2017 11:55:40 -0700 (PDT)"],"DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/relaxed;\n\td=mojatatu-com.20150623.gappssmtp.com; s=20150623;\n\th=mime-version:in-reply-to:references:from:date:message-id:subject:to\n\t:cc; bh=Yjffjz/fvmjlkgg+lCOwc2fpZ0pb54P6Lg0UkMkwnes=;\n\tb=sJGCGqE37G+ATXiDWWnz4ETYiWpuNfTmr8DZsN+8dJVCyMSy73cFdmAspIoY3VFFPa\n\tQc7lpan9SVukHXZb4yxug58VL51xErJeA88VZHV4sIsmKsHGKiaWB0NNHfIyBBTKA4RW\n\tbVEiIFbPesM0RX9qQ56hA4zBx4hQfES6mANILtPR9H3Q4xH2XkqVut/b/wuHhYdgyD/x\n\tHMEnN5dh7U33RnDciQPKnhdP82+oO59Hra0VIN+uksgkRrgGY9ZIPV2CJnX4QK6oiIpT\n\thiyOCWsabwz7dSDv34qbTEIUp8yciHONGefIQC2bZAET1Jgnsaz2fSnUgjKNHQtVL05O\n\t04Bw==","X-Google-DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/relaxed;\n\td=1e100.net; s=20161025;\n\th=x-gm-message-state:mime-version:in-reply-to:references:from:date\n\t:message-id:subject:to:cc;\n\tbh=Yjffjz/fvmjlkgg+lCOwc2fpZ0pb54P6Lg0UkMkwnes=;\n\tb=m/U5dxGGeNO6zs1uordNf7JQFQfkPMtnZYuygHVATzyp2oKwsdDfmKckj2iW8Ntg4z\n\tVtVR+nDqm5Z0iHpuryUZ+w9Ak6srRV9JJh4f4SL7XlHMWy5LHJGbvQCeOicXzHTSAlLW\n\t0L3oRaqToWSMTbqupjMmF/sp7ltF2jo71WeZYnls10HRZj8SkCYC74VeGFnhUBDg2302\n\t8CFbU/z9a4dj+ZaqUO0o+5ga70r8bspgDE58WjqmwrjjprgsNBrCjlYH1T3mCFfpOn3g\n\twfZr22LdT5s7ZlbSjTwz5beFacu3eAVpA5I8dAwxkywbslWqrczC6gNR+RbQJujHQRix\n\tIAvg==","X-Gm-Message-State":"AMCzsaVSOjvdy2K+CqdIhdVIwLETW2J3jjDH0tGod/Q0a7f2iOsmsIQU\n\tih44od73E78TU7+oF4jzjM1eaFXCtA7ZLvOc5xLJ0g==","X-Google-Smtp-Source":"ABhQp+QyJ9mB9nZCSt5PZmNF/sTsBAsG3iufvlri8N2EFZUJCgrXBOuTz5k7vODGeIMgGmkBfGSw4fQSrnQNDulHDh0=","X-Received":"by 10.80.157.141 with SMTP id w13mr4325682ede.151.1509476160427; \n\tTue, 31 Oct 2017 11:56:00 -0700 (PDT)","MIME-Version":"1.0","In-Reply-To":"<0dfff609-d67e-0701-399f-1dbe2ef09bf3@mojatatu.com>","References":"<20171027012443.3306-1-xiyou.wangcong@gmail.com>\n\t\n\t\n\t\n\t\n\t<0dfff609-d67e-0701-399f-1dbe2ef09bf3@mojatatu.com>","From":"Lucas Bates ","Date":"Tue, 31 Oct 2017 14:55:40 -0400","Message-ID":"","Subject":"Re: [Patch net 00/16] net_sched: fix races with RCU callbacks","To":"Jamal Hadi Salim ","Cc":"Cong Wang ,\n\tLinux Kernel Network Developers ,\n\tChris Mi , Daniel Borkmann ,\n\tJiri Pirko , John Fastabend ,\n\t\"Paul E. McKenney\" ","Content-Type":"text/plain; charset=\"UTF-8\"","Sender":"netdev-owner@vger.kernel.org","Precedence":"bulk","List-ID":"","X-Mailing-List":"netdev@vger.kernel.org"}},{"id":1796897,"web_url":"http://patchwork.ozlabs.org/comment/1796897/","msgid":"","list_archive_url":null,"date":"2017-10-31T19:13:29","subject":"Re: [Patch net 00/16] net_sched: fix races with RCU callbacks","submitter":{"id":70959,"url":"http://patchwork.ozlabs.org/api/people/70959/","name":"Lucas Bates","email":"lucasb@mojatatu.com"},"content":"On Tue, Oct 31, 2017 at 2:55 PM, Lucas Bates wrote:\n> On Tue, Oct 31, 2017 at 7:00 AM, Jamal Hadi Salim wrote:\n>> On 17-10-31 01:44 AM, Cong Wang wrote:\n>>>\n>>> Can you try this patch? From your stack trace it is not clear where\n>>> the cause is, but we know that the crash is in __tcf_idr_release(),\n>>> this is how I came up with the following patch:\n>>>\n>>>\n>>> diff --git a/include/net/act_api.h b/include/net/act_api.h\n>>> index b944e0eb93be..5072446d5f06 100644\n>>> --- a/include/net/act_api.h\n>>> +++ b/include/net/act_api.h\n>>> @@ -122,7 +122,9 @@ void tcf_idrinfo_destroy(const struct tc_action_ops\n>>> *ops,\n>>>\n>>> static inline void tc_action_net_exit(struct tc_action_net *tn)\n>>> {\n>>> + rtnl_lock();\n>>> tcf_idrinfo_destroy(tn->ops, tn->idrinfo);\n>>> + rtnl_unlock();\n>>> kfree(tn->idrinfo);\n>>> }\n>>>\n>>>\n>>\n>> Looks like an excellent bet;-> tdc kills the container at the end of the\n>> test.\n>\n> Unfortunately it doesn't seem to have had any effect, I'm still seeing\n> the same bug as yesterday. At Jamal's suggestion I put in a delay\n> after tdc completed running the tests but before it deleted the\n> container - and I didn't run into the bug after many runs. I had no\n> luck getting serial console access on any of our other systems so I\n> have another stack trace photo which I'll send to you directly.\n\nI forgot to mention: it appears the bug appears to be related to\ndeleting the container. Test d052 is the last to run before tdc does\nits teardown and deletes the container; if I add the sleep the bug\nnever gets triggered.","headers":{"Return-Path":"","X-Original-To":"patchwork-incoming@ozlabs.org","Delivered-To":"patchwork-incoming@ozlabs.org","Authentication-Results":["ozlabs.org;\n\tspf=none (mailfrom) smtp.mailfrom=vger.kernel.org\n\t(client-ip=209.132.180.67; helo=vger.kernel.org;\n\tenvelope-from=netdev-owner@vger.kernel.org;\n\treceiver=)","ozlabs.org; dkim=pass (2048-bit key;\n\tunprotected) header.d=mojatatu-com.20150623.gappssmtp.com\n\theader.i=@mojatatu-com.20150623.gappssmtp.com\n\theader.b=\"Xf9e9hYE\"; dkim-atps=neutral"],"Received":["from vger.kernel.org (vger.kernel.org [209.132.180.67])\n\tby ozlabs.org (Postfix) with ESMTP id 3yRLcL31HMz9t3m\n\tfor ;\n\tWed, 1 Nov 2017 06:13:54 +1100 (AEDT)","(majordomo@vger.kernel.org) by vger.kernel.org via listexpand\n\tid S1753852AbdJaTNw (ORCPT );\n\tTue, 31 Oct 2017 15:13:52 -0400","from mail-wm0-f67.google.com ([74.125.82.67]:51732 \"EHLO\n\tmail-wm0-f67.google.com\" rhost-flags-OK-OK-OK-OK) by vger.kernel.org\n\twith ESMTP id S1753797AbdJaTNu (ORCPT\n\t); Tue, 31 Oct 2017 15:13:50 -0400","by mail-wm0-f67.google.com with SMTP id b9so948694wmh.0\n\tfor ; Tue, 31 Oct 2017 12:13:50 -0700 (PDT)","by 10.80.203.204 with HTTP; Tue, 31 Oct 2017 12:13:29 -0700 (PDT)"],"DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/relaxed;\n\td=mojatatu-com.20150623.gappssmtp.com; s=20150623;\n\th=mime-version:in-reply-to:references:from:date:message-id:subject:to\n\t:cc; bh=CUUBdrsxBqMik7WZj1BC8n54kMVVEmr3up7hp+ruxoE=;\n\tb=Xf9e9hYETGpeszt5pUTSS5CuXb3z7iAPs/sRQZX7IsHttQGMiC/oxrostIy2loYaYT\n\taXeRWizz1+mBr5ftkoGNbHp3uAVHjn5wozQ5hHoBJhoxl7V3WbrIK6Cf7BPV0MjmJOuB\n\tpP13zKwBLYIO+4W1Kdm6ge/W6H3R1Y5OAQfdj+eOD3GczKDSnEOMtbCW18eqM5ri18ex\n\t5OlgE1qiAXQXcmcH4cC11dDNVH9Mj4JV2aDwg/Pdk3HE4tviVd1zZuH1AsUIBcG+0XVN\n\tP5l3IAdKyFUAU2T+umH2FUDhd4mhUX0P2o+QolhaJpw8KE8HUpVfPjCJYwwPH7YlO5Z3\n\t8A3w==","X-Google-DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/relaxed;\n\td=1e100.net; s=20161025;\n\th=x-gm-message-state:mime-version:in-reply-to:references:from:date\n\t:message-id:subject:to:cc;\n\tbh=CUUBdrsxBqMik7WZj1BC8n54kMVVEmr3up7hp+ruxoE=;\n\tb=tsFfFf75ghIKZjfm30hfpy3Z74uOrpf/bhRTdxQ86vfr9jaOacJs0B38Cq27dD+OhU\n\tnxqhEO2e3Z+A2gCoQGB8UjlCVW3bY7c1PXYDl/2oNn2GLVfehbkz6kWLqx1xV/tBKosf\n\tNzR37vMHTDP+TnOq8BmSgwpFM/BCVPkGNM2jlrF2+LDp9IinK77PvWVQ5uutjmforWe6\n\tqVrGeIGsvTi1ZIBgDCg0ohPXoQE4IFNa7v67b/u90j0E10JimX46pFHb34bSFDOOsibU\n\tWS737Kg4o7Eb6kZH7O7iTpLyOKUdq1d3qaNgfCC2mhrag7xRJBAInpSt3D7PJzpcxq8R\n\t/7Ug==","X-Gm-Message-State":"AMCzsaWBqOwTAuwN3CEKEGXDJt3/eXiVell4AB1AqwDkkQ8Q1psQiqRs\n\tGJ7CZM18qflEJaS9N8FJj3qfczyNnry8SFM0ABX0EQ==","X-Google-Smtp-Source":"ABhQp+Srvqw7eAVNczwFu7ihB7UhKcClv709tpVCrQzFmEdlpNgVkxoMjnZKYy25Hk//QbEYgWyB5TUjtF6CJTz9wDM=","X-Received":"by 10.80.154.35 with SMTP id o32mr4143888edb.125.1509477229608; \n\tTue, 31 Oct 2017 12:13:49 -0700 (PDT)","MIME-Version":"1.0","In-Reply-To":"","References":"<20171027012443.3306-1-xiyou.wangcong@gmail.com>\n\t\n\t\n\t\n\t\n\t<0dfff609-d67e-0701-399f-1dbe2ef09bf3@mojatatu.com>\n\t","From":"Lucas Bates ","Date":"Tue, 31 Oct 2017 15:13:29 -0400","Message-ID":"","Subject":"Re: [Patch net 00/16] net_sched: fix races with RCU callbacks","To":"Jamal Hadi Salim ","Cc":"Cong Wang ,\n\tLinux Kernel Network Developers ,\n\tChris Mi , Daniel Borkmann ,\n\tJiri Pirko , John Fastabend ,\n\t\"Paul E. McKenney\" ","Content-Type":"text/plain; charset=\"UTF-8\"","Sender":"netdev-owner@vger.kernel.org","Precedence":"bulk","List-ID":"","X-Mailing-List":"netdev@vger.kernel.org"}},{"id":1796977,"web_url":"http://patchwork.ozlabs.org/comment/1796977/","msgid":"","list_archive_url":null,"date":"2017-10-31T22:09:57","subject":"Re: [Patch net 00/16] net_sched: fix races with RCU callbacks","submitter":{"id":211,"url":"http://patchwork.ozlabs.org/api/people/211/","name":"Cong Wang","email":"xiyou.wangcong@gmail.com"},"content":"On Tue, Oct 31, 2017 at 12:13 PM, Lucas Bates wrote:\n> On Tue, Oct 31, 2017 at 2:55 PM, Lucas Bates wrote:\n>> Unfortunately it doesn't seem to have had any effect, I'm still seeing\n>> the same bug as yesterday. At Jamal's suggestion I put in a delay\n\nAs I replied to you privately, it is probably because idrinfo is freed\nbefore action itself. So just RTNL is not enough.\n\n\n>> after tdc completed running the tests but before it deleted the\n>> container - and I didn't run into the bug after many runs. I had no\n>> luck getting serial console access on any of our other systems so I\n>> have another stack trace photo which I'll send to you directly.\n>\n> I forgot to mention: it appears the bug appears to be related to\n> deleting the container. Test d052 is the last to run before tdc does\n> its teardown and deletes the container; if I add the sleep the bug\n> never gets triggered.\n\nThis almost rules out the guilty of this patchset.\n\nI will provide a patch for you to test, since I can't reproduce it here.\n\nThanks!","headers":{"Return-Path":"","X-Original-To":"patchwork-incoming@ozlabs.org","Delivered-To":"patchwork-incoming@ozlabs.org","Authentication-Results":["ozlabs.org;\n\tspf=none (mailfrom) smtp.mailfrom=vger.kernel.org\n\t(client-ip=209.132.180.67; helo=vger.kernel.org;\n\tenvelope-from=netdev-owner@vger.kernel.org;\n\treceiver=)","ozlabs.org; dkim=pass (2048-bit key;\n\tunprotected) header.d=gmail.com header.i=@gmail.com\n\theader.b=\"EBl3HtNQ\"; dkim-atps=neutral"],"Received":["from vger.kernel.org (vger.kernel.org [209.132.180.67])\n\tby ozlabs.org (Postfix) with ESMTP id 3yRQWy1QFGz9t41\n\tfor ;\n\tWed, 1 Nov 2017 09:10:22 +1100 (AEDT)","(majordomo@vger.kernel.org) by vger.kernel.org via listexpand\n\tid S932341AbdJaWKU (ORCPT );\n\tTue, 31 Oct 2017 18:10:20 -0400","from mail-pf0-f196.google.com ([209.85.192.196]:43544 \"EHLO\n\tmail-pf0-f196.google.com\" rhost-flags-OK-OK-OK-OK) by vger.kernel.org\n\twith ESMTP id S1753486AbdJaWKT (ORCPT\n\t); Tue, 31 Oct 2017 18:10:19 -0400","by mail-pf0-f196.google.com with SMTP id a8so364632pfc.0\n\tfor ; Tue, 31 Oct 2017 15:10:18 -0700 (PDT)","by 10.100.170.205 with HTTP; Tue, 31 Oct 2017 15:09:57 -0700 (PDT)"],"DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/relaxed;\n\td=gmail.com; s=20161025;\n\th=mime-version:in-reply-to:references:from:date:message-id:subject:to\n\t:cc; bh=lxLG3A2j3ADBRR7K97dvBjoGkxGZnv/jROgaN9kOIbY=;\n\tb=EBl3HtNQqqooO2qYd7/02O6gLdxMmpwu75l8uKfGmwEKGaXLBJt7eFX6HE6sLSB44J\n\tT+W87dcRIjfgh4aJRb2B4Te8dWX+ETXQFHxl1j/9uhHpM9dZQ7e3QRmjxUsjXGLU8GO1\n\trVCZe7h20/eM7pN6eOLxuKYWJu/IlkKJ96V9uOTV0k4bLkcsROTehiiNeAB7hlVidvD2\n\tdLwuTNM0b/MpOC9YJ5WCmLTkjPtnkYr6VfoNCZOdnbL6p5qKIxeijx9ArIkstxt8en43\n\tjHAJYc2SwdPEE556xoRNf4Yw3voqDRc7dxNJ0UJ8htbzVh2/x4yKc7Y4HtOn7iL+JWaJ\n\toA9A==","X-Google-DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/relaxed;\n\td=1e100.net; s=20161025;\n\th=x-gm-message-state:mime-version:in-reply-to:references:from:date\n\t:message-id:subject:to:cc;\n\tbh=lxLG3A2j3ADBRR7K97dvBjoGkxGZnv/jROgaN9kOIbY=;\n\tb=HxBKFyQwszVQA1qSomXnJoWlzxG+NCumuANV72dc/gOjGX7X6bGJH8CpSzMviPrKg3\n\tMggDNdMC0t4LNYLEHF4+VwmNvcb1WXuC2bKyTylqhv+iiNfcmC4BLQ/CHS7pfy/JY7bw\n\tEruF1P9kSIFOU1oXI0dRqeFm+mDMusedrPs6lVxnhXvEuDUtB0xdVh48TtSZISaCPiz2\n\tyZrLgb+FbjiCKHuycKc4XS4QMOAlrQd/C2OsLn0tA2twzn8yFW37ukggpoZwizlsTrTW\n\tOb43neN35/Ev6NyUtzCbtXr4G7dOWuzfogj2/ZU4cZ5RAMFeTiBLgqzoPrApls86J9Pp\n\tsetA==","X-Gm-Message-State":"AMCzsaURfbzPFjMDezgxXYXCBb/J51WliQoo996bN05i2jacqxwFysHh\n\tUa626aEj1fUBgNR8S22yT5LrqCfb7q7kF84xJbM=","X-Google-Smtp-Source":"ABhQp+QU22AHFtB+vOCPTfzJ6d3Rz7EBXWu04Bvzs7KDxAMh0BlPMStllrjQqfy4+qsvo/3Em6ScQqfZyg8pmOjEHpA=","X-Received":"by 10.99.43.71 with SMTP id r68mr3405519pgr.348.1509487818308;\n\tTue, 31 Oct 2017 15:10:18 -0700 (PDT)","MIME-Version":"1.0","In-Reply-To":"","References":"<20171027012443.3306-1-xiyou.wangcong@gmail.com>\n\t\n\t\n\t\n\t\n\t<0dfff609-d67e-0701-399f-1dbe2ef09bf3@mojatatu.com>\n\t\n\t","From":"Cong Wang ","Date":"Tue, 31 Oct 2017 15:09:57 -0700","Message-ID":"","Subject":"Re: [Patch net 00/16] net_sched: fix races with RCU callbacks","To":"Lucas Bates ","Cc":"Jamal Hadi Salim ,\n\tLinux Kernel Network Developers ,\n\tChris Mi , Daniel Borkmann ,\n\tJiri Pirko , John Fastabend ,\n\t\"Paul E. McKenney\" ","Content-Type":"text/plain; charset=\"UTF-8\"","Sender":"netdev-owner@vger.kernel.org","Precedence":"bulk","List-ID":"","X-Mailing-List":"netdev@vger.kernel.org"}},{"id":1796993,"web_url":"http://patchwork.ozlabs.org/comment/1796993/","msgid":"","list_archive_url":null,"date":"2017-10-31T23:02:17","subject":"Re: [Patch net 00/16] net_sched: fix races with RCU callbacks","submitter":{"id":211,"url":"http://patchwork.ozlabs.org/api/people/211/","name":"Cong Wang","email":"xiyou.wangcong@gmail.com"},"content":"On Tue, Oct 31, 2017 at 3:09 PM, Cong Wang wrote:\n>\n> This almost rules out the guilty of this patchset.\n>\n> I will provide a patch for you to test, since I can't reproduce it here.\n>\n\nLucas, please test the attached patch, it applies to latest -net.\n\nNote, it is a combination of 3 patches which together close the\nuse-after-free you reported here, I hope.\n\nPlease let me know if this works. My basic tests run well without\nany stack traces or memory leaks.\n\nThanks!\ndiff --git a/include/net/act_api.h b/include/net/act_api.h\nindex b944e0eb93be..c6c8129c3647 100644\n--- a/include/net/act_api.h\n+++ b/include/net/act_api.h\n@@ -13,6 +13,7 @@\n struct tcf_idrinfo {\n \tspinlock_t\tlock;\n \tstruct idr\taction_idr;\n+\tstruct net\t*net;\n };\n \n struct tc_action_ops;\n@@ -100,11 +101,12 @@ struct tc_action_ops {\n struct tc_action_net {\n \tstruct tcf_idrinfo *idrinfo;\n \tconst struct tc_action_ops *ops;\n+\tstruct net *net;\n };\n \n static inline\n int tc_action_net_init(struct tc_action_net *tn,\n-\t\t const struct tc_action_ops *ops)\n+\t\t const struct tc_action_ops *ops, struct net *net)\n {\n \tint err = 0;\n \n@@ -112,6 +114,7 @@ int tc_action_net_init(struct tc_action_net *tn,\n \tif (!tn->idrinfo)\n \t\treturn -ENOMEM;\n \ttn->ops = ops;\n+\ttn->idrinfo->net = net;\n \tspin_lock_init(&tn->idrinfo->lock);\n \tidr_init(&tn->idrinfo->action_idr);\n \treturn err;\n@@ -122,7 +125,9 @@ void tcf_idrinfo_destroy(const struct tc_action_ops *ops,\n \n static inline void tc_action_net_exit(struct tc_action_net *tn)\n {\n+\trtnl_lock();\n \ttcf_idrinfo_destroy(tn->ops, tn->idrinfo);\n+\trtnl_unlock();\n \tkfree(tn->idrinfo);\n }\n \ndiff --git a/net/sched/act_api.c b/net/sched/act_api.c\nindex da6fa82c98a8..241bf3f925de 100644\n--- a/net/sched/act_api.c\n+++ b/net/sched/act_api.c\n@@ -78,6 +78,7 @@ static void tcf_idr_remove(struct tcf_idrinfo *idrinfo, struct tc_action *p)\n \tspin_lock_bh(&idrinfo->lock);\n \tidr_remove_ext(&idrinfo->action_idr, p->tcfa_index);\n \tspin_unlock_bh(&idrinfo->lock);\n+\tput_net(idrinfo->net);\n \tgen_kill_estimator(&p->tcfa_rate_est);\n \tfree_tcf(p);\n }\n@@ -86,6 +87,8 @@ int __tcf_idr_release(struct tc_action *p, bool bind, bool strict)\n {\n \tint ret = 0;\n \n+\tASSERT_RTNL();\n+\n \tif (p) {\n \t\tif (bind)\n \t\t\tp->tcfa_bindcnt--;\n@@ -93,7 +96,7 @@ int __tcf_idr_release(struct tc_action *p, bool bind, bool strict)\n \t\t\treturn -EPERM;\n \n \t\tp->tcfa_refcnt--;\n-\t\tif (p->tcfa_bindcnt <= 0 && p->tcfa_refcnt <= 0) {\n+\t\tif (p->tcfa_bindcnt == 0 && p->tcfa_refcnt == 0) {\n \t\t\tif (p->ops->cleanup)\n \t\t\t\tp->ops->cleanup(p, bind);\n \t\t\ttcf_idr_remove(p->idrinfo, p);\n@@ -334,6 +337,7 @@ int tcf_idr_create(struct tc_action_net *tn, u32 index, struct nlattr *est,\n \tp->idrinfo = idrinfo;\n \tp->ops = ops;\n \tINIT_LIST_HEAD(&p->list);\n+\tget_net(idrinfo->net);\n \t*a = p;\n \treturn 0;\n }\ndiff --git a/net/sched/act_bpf.c b/net/sched/act_bpf.c\nindex c0c707eb2c96..9bce8cc84cbb 100644\n--- a/net/sched/act_bpf.c\n+++ b/net/sched/act_bpf.c\n@@ -398,7 +398,7 @@ static __net_init int bpf_init_net(struct net *net)\n {\n \tstruct tc_action_net *tn = net_generic(net, bpf_net_id);\n \n-\treturn tc_action_net_init(tn, &act_bpf_ops);\n+\treturn tc_action_net_init(tn, &act_bpf_ops, net);\n }\n \n static void __net_exit bpf_exit_net(struct net *net)\ndiff --git a/net/sched/act_connmark.c b/net/sched/act_connmark.c\nindex 10b7a8855a6c..34e52d01a5dd 100644\n--- a/net/sched/act_connmark.c\n+++ b/net/sched/act_connmark.c\n@@ -206,7 +206,7 @@ static __net_init int connmark_init_net(struct net *net)\n {\n \tstruct tc_action_net *tn = net_generic(net, connmark_net_id);\n \n-\treturn tc_action_net_init(tn, &act_connmark_ops);\n+\treturn tc_action_net_init(tn, &act_connmark_ops, net);\n }\n \n static void __net_exit connmark_exit_net(struct net *net)\ndiff --git a/net/sched/act_csum.c b/net/sched/act_csum.c\nindex 1c40caadcff9..35171df2ebef 100644\n--- a/net/sched/act_csum.c\n+++ b/net/sched/act_csum.c\n@@ -626,7 +626,7 @@ static __net_init int csum_init_net(struct net *net)\n {\n \tstruct tc_action_net *tn = net_generic(net, csum_net_id);\n \n-\treturn tc_action_net_init(tn, &act_csum_ops);\n+\treturn tc_action_net_init(tn, &act_csum_ops, net);\n }\n \n static void __net_exit csum_exit_net(struct net *net)\ndiff --git a/net/sched/act_gact.c b/net/sched/act_gact.c\nindex e29a48ef7fc3..ef7f7f39d26d 100644\n--- a/net/sched/act_gact.c\n+++ b/net/sched/act_gact.c\n@@ -232,7 +232,7 @@ static __net_init int gact_init_net(struct net *net)\n {\n \tstruct tc_action_net *tn = net_generic(net, gact_net_id);\n \n-\treturn tc_action_net_init(tn, &act_gact_ops);\n+\treturn tc_action_net_init(tn, &act_gact_ops, net);\n }\n \n static void __net_exit gact_exit_net(struct net *net)\ndiff --git a/net/sched/act_ife.c b/net/sched/act_ife.c\nindex 8ccd35825b6b..f65e4b5058e0 100644\n--- a/net/sched/act_ife.c\n+++ b/net/sched/act_ife.c\n@@ -818,7 +818,7 @@ static __net_init int ife_init_net(struct net *net)\n {\n \tstruct tc_action_net *tn = net_generic(net, ife_net_id);\n \n-\treturn tc_action_net_init(tn, &act_ife_ops);\n+\treturn tc_action_net_init(tn, &act_ife_ops, net);\n }\n \n static void __net_exit ife_exit_net(struct net *net)\ndiff --git a/net/sched/act_ipt.c b/net/sched/act_ipt.c\nindex d9e399a7e3d5..dbdf3b2470d5 100644\n--- a/net/sched/act_ipt.c\n+++ b/net/sched/act_ipt.c\n@@ -334,7 +334,7 @@ static __net_init int ipt_init_net(struct net *net)\n {\n \tstruct tc_action_net *tn = net_generic(net, ipt_net_id);\n \n-\treturn tc_action_net_init(tn, &act_ipt_ops);\n+\treturn tc_action_net_init(tn, &act_ipt_ops, net);\n }\n \n static void __net_exit ipt_exit_net(struct net *net)\n@@ -384,7 +384,7 @@ static __net_init int xt_init_net(struct net *net)\n {\n \tstruct tc_action_net *tn = net_generic(net, xt_net_id);\n \n-\treturn tc_action_net_init(tn, &act_xt_ops);\n+\treturn tc_action_net_init(tn, &act_xt_ops, net);\n }\n \n static void __net_exit xt_exit_net(struct net *net)\ndiff --git a/net/sched/act_mirred.c b/net/sched/act_mirred.c\nindex 416627c66f08..84759cfd5a33 100644\n--- a/net/sched/act_mirred.c\n+++ b/net/sched/act_mirred.c\n@@ -343,7 +343,7 @@ static __net_init int mirred_init_net(struct net *net)\n {\n \tstruct tc_action_net *tn = net_generic(net, mirred_net_id);\n \n-\treturn tc_action_net_init(tn, &act_mirred_ops);\n+\treturn tc_action_net_init(tn, &act_mirred_ops, net);\n }\n \n static void __net_exit mirred_exit_net(struct net *net)\ndiff --git a/net/sched/act_nat.c b/net/sched/act_nat.c\nindex c365d01b99c8..7eeaaf9217b6 100644\n--- a/net/sched/act_nat.c\n+++ b/net/sched/act_nat.c\n@@ -307,7 +307,7 @@ static __net_init int nat_init_net(struct net *net)\n {\n \tstruct tc_action_net *tn = net_generic(net, nat_net_id);\n \n-\treturn tc_action_net_init(tn, &act_nat_ops);\n+\treturn tc_action_net_init(tn, &act_nat_ops, net);\n }\n \n static void __net_exit nat_exit_net(struct net *net)\ndiff --git a/net/sched/act_pedit.c b/net/sched/act_pedit.c\nindex 491fe5deb09e..b3d82c334a5f 100644\n--- a/net/sched/act_pedit.c\n+++ b/net/sched/act_pedit.c\n@@ -450,7 +450,7 @@ static __net_init int pedit_init_net(struct net *net)\n {\n \tstruct tc_action_net *tn = net_generic(net, pedit_net_id);\n \n-\treturn tc_action_net_init(tn, &act_pedit_ops);\n+\treturn tc_action_net_init(tn, &act_pedit_ops, net);\n }\n \n static void __net_exit pedit_exit_net(struct net *net)\ndiff --git a/net/sched/act_police.c b/net/sched/act_police.c\nindex 3bb2ebf9e9ae..9ec42b26e4b9 100644\n--- a/net/sched/act_police.c\n+++ b/net/sched/act_police.c\n@@ -331,7 +331,7 @@ static __net_init int police_init_net(struct net *net)\n {\n \tstruct tc_action_net *tn = net_generic(net, police_net_id);\n \n-\treturn tc_action_net_init(tn, &act_police_ops);\n+\treturn tc_action_net_init(tn, &act_police_ops, net);\n }\n \n static void __net_exit police_exit_net(struct net *net)\ndiff --git a/net/sched/act_sample.c b/net/sched/act_sample.c\nindex a9f9a2ccc664..71b8405e9718 100644\n--- a/net/sched/act_sample.c\n+++ b/net/sched/act_sample.c\n@@ -240,7 +240,7 @@ static __net_init int sample_init_net(struct net *net)\n {\n \tstruct tc_action_net *tn = net_generic(net, sample_net_id);\n \n-\treturn tc_action_net_init(tn, &act_sample_ops);\n+\treturn tc_action_net_init(tn, &act_sample_ops, net);\n }\n \n static void __net_exit sample_exit_net(struct net *net)\ndiff --git a/net/sched/act_simple.c b/net/sched/act_simple.c\nindex e7b57e5071a3..a8d0ea95f894 100644\n--- a/net/sched/act_simple.c\n+++ b/net/sched/act_simple.c\n@@ -201,7 +201,7 @@ static __net_init int simp_init_net(struct net *net)\n {\n \tstruct tc_action_net *tn = net_generic(net, simp_net_id);\n \n-\treturn tc_action_net_init(tn, &act_simp_ops);\n+\treturn tc_action_net_init(tn, &act_simp_ops, net);\n }\n \n static void __net_exit simp_exit_net(struct net *net)\ndiff --git a/net/sched/act_skbedit.c b/net/sched/act_skbedit.c\nindex 59949d61f20d..fbac62472e09 100644\n--- a/net/sched/act_skbedit.c\n+++ b/net/sched/act_skbedit.c\n@@ -238,7 +238,7 @@ static __net_init int skbedit_init_net(struct net *net)\n {\n \tstruct tc_action_net *tn = net_generic(net, skbedit_net_id);\n \n-\treturn tc_action_net_init(tn, &act_skbedit_ops);\n+\treturn tc_action_net_init(tn, &act_skbedit_ops, net);\n }\n \n static void __net_exit skbedit_exit_net(struct net *net)\ndiff --git a/net/sched/act_skbmod.c b/net/sched/act_skbmod.c\nindex b642ad3d39dd..8e12d8897d2f 100644\n--- a/net/sched/act_skbmod.c\n+++ b/net/sched/act_skbmod.c\n@@ -263,7 +263,7 @@ static __net_init int skbmod_init_net(struct net *net)\n {\n \tstruct tc_action_net *tn = net_generic(net, skbmod_net_id);\n \n-\treturn tc_action_net_init(tn, &act_skbmod_ops);\n+\treturn tc_action_net_init(tn, &act_skbmod_ops, net);\n }\n \n static void __net_exit skbmod_exit_net(struct net *net)\ndiff --git a/net/sched/act_tunnel_key.c b/net/sched/act_tunnel_key.c\nindex 30c96274c638..c33faa373cf2 100644\n--- a/net/sched/act_tunnel_key.c\n+++ b/net/sched/act_tunnel_key.c\n@@ -322,7 +322,7 @@ static __net_init int tunnel_key_init_net(struct net *net)\n {\n \tstruct tc_action_net *tn = net_generic(net, tunnel_key_net_id);\n \n-\treturn tc_action_net_init(tn, &act_tunnel_key_ops);\n+\treturn tc_action_net_init(tn, &act_tunnel_key_ops, net);\n }\n \n static void __net_exit tunnel_key_exit_net(struct net *net)\ndiff --git a/net/sched/act_vlan.c b/net/sched/act_vlan.c\nindex 16eb067a8d8f..115fc33cc6d8 100644\n--- a/net/sched/act_vlan.c\n+++ b/net/sched/act_vlan.c\n@@ -269,7 +269,7 @@ static __net_init int vlan_init_net(struct net *net)\n {\n \tstruct tc_action_net *tn = net_generic(net, vlan_net_id);\n \n-\treturn tc_action_net_init(tn, &act_vlan_ops);\n+\treturn tc_action_net_init(tn, &act_vlan_ops, net);\n }\n \n static void __net_exit vlan_exit_net(struct net *net)","headers":{"Return-Path":"","X-Original-To":"patchwork-incoming@ozlabs.org","Delivered-To":"patchwork-incoming@ozlabs.org","Authentication-Results":["ozlabs.org;\n\tspf=none (mailfrom) smtp.mailfrom=vger.kernel.org\n\t(client-ip=209.132.180.67; helo=vger.kernel.org;\n\tenvelope-from=netdev-owner@vger.kernel.org;\n\treceiver=)","ozlabs.org; dkim=pass (2048-bit key;\n\tunprotected) header.d=gmail.com header.i=@gmail.com\n\theader.b=\"sKIDUm47\"; dkim-atps=neutral"],"Received":["from vger.kernel.org (vger.kernel.org [209.132.180.67])\n\tby ozlabs.org (Postfix) with ESMTP id 3yRRhL6DqJz9t3C\n\tfor ;\n\tWed, 1 Nov 2017 10:02:42 +1100 (AEDT)","(majordomo@vger.kernel.org) by vger.kernel.org via listexpand\n\tid S1753960AbdJaXCk (ORCPT );\n\tTue, 31 Oct 2017 19:02:40 -0400","from mail-pf0-f170.google.com ([209.85.192.170]:56980 \"EHLO\n\tmail-pf0-f170.google.com\" rhost-flags-OK-OK-OK-OK) by vger.kernel.org\n\twith ESMTP id S1752473AbdJaXCi (ORCPT\n\t); Tue, 31 Oct 2017 19:02:38 -0400","by mail-pf0-f170.google.com with SMTP id b85so433924pfj.13\n\tfor ; Tue, 31 Oct 2017 16:02:38 -0700 (PDT)","by 10.100.170.205 with HTTP; Tue, 31 Oct 2017 16:02:17 -0700 (PDT)"],"DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/relaxed;\n\td=gmail.com; s=20161025;\n\th=mime-version:in-reply-to:references:from:date:message-id:subject:to\n\t:cc; bh=vhgKnDVvocKZc24GAyfnys+gUUhPM+C+0F9GUpV+k4k=;\n\tb=sKIDUm47tCUPMrOAMusNOxfl+Bo3TokJtSqPKdo4YR5LoVXaXW5pf4zq8IuyAA1VbI\n\tDjr2x2PrOJF8OASbktwHmR3UEkrU/0+3FiFEvzZg+5EynWy0+eX335oTu44WksE95YI5\n\tLTsiWqZLeC3nYuUb5H4T+6H1z+1ayjQragDS3nOh4gm3CqNU+jBHa/wDsMYz+XTuV68F\n\tvpicL7iApU+ogdn+1HaNOFVWX7Rwe5gmNAG1cPdp+FTfVhg37467x2ziYWMJQjwErynp\n\tKabrm/SNiYE4HBVZLmbosyTZ6Ie/9Vqkr96XOcTBFb8L8fGhL1K8afwpTsxSic8eBNcu\n\tyzoA==","X-Google-DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/relaxed;\n\td=1e100.net; s=20161025;\n\th=x-gm-message-state:mime-version:in-reply-to:references:from:date\n\t:message-id:subject:to:cc;\n\tbh=vhgKnDVvocKZc24GAyfnys+gUUhPM+C+0F9GUpV+k4k=;\n\tb=q8KNtiOizcAcyiURBPzog83ETF6/VCDGfslob08wnuox51Ep2CCO3xWIsyQ6zTQuL5\n\t6qR9ao7+sh0UjH17S6Rxj3VISZnVrRHl0mp1A5vcFUuNTBngO5egozcwlzK4Yxj9+3xh\n\tzOrapD38nWD0399bnG/KYTx2Px1K3qeubDLbyoqMsnLxOaU3UNNPa77qoI7wfwYrW3Gj\n\tyss/Wp0B5wSpWK3aJAS1zMqzNOU0WUuxCVqW/uxeCmVViHSMYgNST35Tite7pBGaD3cm\n\tu+FVhM3AFIzBu8JTBFZaHENtTxLql2X4aEMh7djMYfEqunZsrFrTgNPusKC+FSyv2uLe\n\tJj8A==","X-Gm-Message-State":"AMCzsaUj+oVehJldX7YbQKsuGY0y096WPNdioX5/yqyKG1rPBOD+Fgnc\n\tA5+qmo+IMqCv3LGmr8RnAGGoeHhSirDLaztzFnqLfQ==","X-Google-Smtp-Source":"ABhQp+QYkwWq0RMKmQ30RyBVosmLUtCiwuSo/X2S8qCiddkWbqCdz5scmRIrD8G/xsIpnBJrGcrw4mD8kTE/ZOuXua4=","X-Received":"by 10.98.93.12 with SMTP id r12mr3838882pfb.286.1509490958201;\n\tTue, 31 Oct 2017 16:02:38 -0700 (PDT)","MIME-Version":"1.0","In-Reply-To":"","References":"<20171027012443.3306-1-xiyou.wangcong@gmail.com>\n\t\n\t\n\t\n\t\n\t<0dfff609-d67e-0701-399f-1dbe2ef09bf3@mojatatu.com>\n\t\n\t\n\t","From":"Cong Wang ","Date":"Tue, 31 Oct 2017 16:02:17 -0700","Message-ID":"","Subject":"Re: [Patch net 00/16] net_sched: fix races with RCU callbacks","To":"Lucas Bates ","Cc":"Jamal Hadi Salim ,\n\tLinux Kernel Network Developers ,\n\tChris Mi , Daniel Borkmann ,\n\tJiri Pirko , John Fastabend ,\n\t\"Paul E. McKenney\" ","Content-Type":"multipart/mixed; boundary=\"001a113edcea5b74cc055cdfc04b\"","Sender":"netdev-owner@vger.kernel.org","Precedence":"bulk","List-ID":"","X-Mailing-List":"netdev@vger.kernel.org"}},{"id":1797370,"web_url":"http://patchwork.ozlabs.org/comment/1797370/","msgid":"","list_archive_url":null,"date":"2017-11-01T16:55:45","subject":"Re: [Patch net 00/16] net_sched: fix races with RCU callbacks","submitter":{"id":70959,"url":"http://patchwork.ozlabs.org/api/people/70959/","name":"Lucas Bates","email":"lucasb@mojatatu.com"},"content":"On Tue, Oct 31, 2017 at 7:02 PM, Cong Wang wrote:\n> On Tue, Oct 31, 2017 at 3:09 PM, Cong Wang wrote:\n>>\n>> This almost rules out the guilty of this patchset.\n>>\n>> I will provide a patch for you to test, since I can't reproduce it here.\n>>\n>\n> Lucas, please test the attached patch, it applies to latest -net.\n>\n> Note, it is a combination of 3 patches which together close the\n> use-after-free you reported here, I hope.\n>\n> Please let me know if this works. My basic tests run well without\n> any stack traces or memory leaks.\n\nThe results look good to me.\n\nI compiled with the patch and ran the test again in a loop. I set it\nto run 50 times in a row and didn't trigger the bug once (previous\nrecord was 4 times).","headers":{"Return-Path":"","X-Original-To":"patchwork-incoming@ozlabs.org","Delivered-To":"patchwork-incoming@ozlabs.org","Authentication-Results":["ozlabs.org;\n\tspf=none (mailfrom) smtp.mailfrom=vger.kernel.org\n\t(client-ip=209.132.180.67; helo=vger.kernel.org;\n\tenvelope-from=netdev-owner@vger.kernel.org;\n\treceiver=)","ozlabs.org; dkim=pass (2048-bit key;\n\tunprotected) header.d=mojatatu-com.20150623.gappssmtp.com\n\theader.i=@mojatatu-com.20150623.gappssmtp.com\n\theader.b=\"cHNg7xFW\"; dkim-atps=neutral"],"Received":["from vger.kernel.org (vger.kernel.org [209.132.180.67])\n\tby ozlabs.org (Postfix) with ESMTP id 3yRvVz43RXz9sPs\n\tfor ;\n\tThu, 2 Nov 2017 03:56:11 +1100 (AEDT)","(majordomo@vger.kernel.org) by vger.kernel.org via listexpand\n\tid S1754932AbdKAQ4J (ORCPT );\n\tWed, 1 Nov 2017 12:56:09 -0400","from mail-wm0-f50.google.com ([74.125.82.50]:49733 \"EHLO\n\tmail-wm0-f50.google.com\" rhost-flags-OK-OK-OK-OK) by vger.kernel.org\n\twith ESMTP id S1754773AbdKAQ4H (ORCPT\n\t); Wed, 1 Nov 2017 12:56:07 -0400","by mail-wm0-f50.google.com with SMTP id b189so5914053wmd.4\n\tfor ; Wed, 01 Nov 2017 09:56:07 -0700 (PDT)","by 10.80.203.204 with HTTP; Wed, 1 Nov 2017 09:55:45 -0700 (PDT)"],"DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/relaxed;\n\td=mojatatu-com.20150623.gappssmtp.com; s=20150623;\n\th=mime-version:in-reply-to:references:from:date:message-id:subject:to\n\t:cc; bh=NmQ2maOXlj2mAYwVfLlSUzjPvqiEc4kBcNKEEOTYe9U=;\n\tb=cHNg7xFWk1c4E8Po8QFyztD9xjA/26rI23ooY07qCzlpecOsZsqRWvC4seMEHayBgA\n\tofw1CIcRkUL+I+keH5a953h1ZRiOl4T7h1F3+DYLuJTAO1SgtkH+HNRP6vpp45m48ol/\n\tsNU7NAQCMNeUa0C//qpcA9B67urPYgiJnnqanImLEgL1PPUX/jxFWbPaIOfb+yje5zkT\n\trl+omRJ+ONF4Pmc/G8ec2TE/76/hCMzKwL8nRLKzaeYyvv/xmaduGfsmrkOuDz9hNmLH\n\t4YBuhaYgR/cpvg3adESNTN8X1is348gWcJMpZylVW8544y068OYZk76PFs7bz0WNLvBO\n\tNoFQ==","X-Google-DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/relaxed;\n\td=1e100.net; s=20161025;\n\th=x-gm-message-state:mime-version:in-reply-to:references:from:date\n\t:message-id:subject:to:cc;\n\tbh=NmQ2maOXlj2mAYwVfLlSUzjPvqiEc4kBcNKEEOTYe9U=;\n\tb=T0Yuwwio+JksS+iALpnJLbFlLN+LoWVO9MpHqkwn3CeEiOLzTPdxCfiVO4BuizSOd1\n\tvv04I9EhJHm+JCz/jVtyJzf8o7rJpf2ho8PV0mL8CVBQHexkGwtVF4O/ynBFQ7XNvfPK\n\tNuqTwbcJOcaRJNJU72Ssg7n+jlPBnnpXqqDWBXSrTFVeTdTgF+8EDT1f1sAa2QAoRF2R\n\tqLzt2ZO0RHzy8tlKL33WdqkoWW7/B7Yacyj6QWjhPwsNl30yAwty/4IYV7shjkyUtZwk\n\tT2HbYLL3FAQ0T4tOZ0GoCeEak7M4FH+RJuxtbISQsF8lnEhwGl+PQRSC0x2mUm2uzThk\n\tRznA==","X-Gm-Message-State":"AMCzsaUPrsKtarb7moQLbmw47UnTEEr2Qp4E/6IW2hCH6kOalqz8j1Kq\n\tuCytl5CxbGTyvRDyNliTy3JufEyMtzhwb4gLHq/U0g==","X-Google-Smtp-Source":"ABhQp+TCa4Qa5CfayDXDh+fsZn+eDx6SmLA8jROaE6NDaYiWn7yuIvMcN9zCI5kCSELFb7vwaNeypV/8WcpBgdSYFUU=","X-Received":"by 10.80.137.91 with SMTP id f27mr1075591edf.18.1509555366337;\n\tWed, 01 Nov 2017 09:56:06 -0700 (PDT)","MIME-Version":"1.0","In-Reply-To":"","References":"<20171027012443.3306-1-xiyou.wangcong@gmail.com>\n\t\n\t\n\t\n\t\n\t<0dfff609-d67e-0701-399f-1dbe2ef09bf3@mojatatu.com>\n\t\n\t\n\t\n\t","From":"Lucas Bates ","Date":"Wed, 1 Nov 2017 12:55:45 -0400","Message-ID":"","Subject":"Re: [Patch net 00/16] net_sched: fix races with RCU callbacks","To":"Cong Wang ","Cc":"Jamal Hadi Salim ,\n\tLinux Kernel Network Developers ,\n\tChris Mi , Daniel Borkmann ,\n\tJiri Pirko , John Fastabend ,\n\t\"Paul E. McKenney\" ","Content-Type":"text/plain; charset=\"UTF-8\"","Sender":"netdev-owner@vger.kernel.org","Precedence":"bulk","List-ID":"","X-Mailing-List":"netdev@vger.kernel.org"}},{"id":1797375,"web_url":"http://patchwork.ozlabs.org/comment/1797375/","msgid":"","list_archive_url":null,"date":"2017-11-01T16:59:04","subject":"Re: [Patch net 00/16] net_sched: fix races with RCU callbacks","submitter":{"id":211,"url":"http://patchwork.ozlabs.org/api/people/211/","name":"Cong Wang","email":"xiyou.wangcong@gmail.com"},"content":"On Wed, Nov 1, 2017 at 9:55 AM, Lucas Bates wrote:\n> On Tue, Oct 31, 2017 at 7:02 PM, Cong Wang wrote:\n>> On Tue, Oct 31, 2017 at 3:09 PM, Cong Wang wrote:\n>>>\n>>> This almost rules out the guilty of this patchset.\n>>>\n>>> I will provide a patch for you to test, since I can't reproduce it here.\n>>>\n>>\n>> Lucas, please test the attached patch, it applies to latest -net.\n>>\n>> Note, it is a combination of 3 patches which together close the\n>> use-after-free you reported here, I hope.\n>>\n>> Please let me know if this works. My basic tests run well without\n>> any stack traces or memory leaks.\n>\n> The results look good to me.\n>\n> I compiled with the patch and ran the test again in a loop. I set it\n> to run 50 times in a row and didn't trigger the bug once (previous\n> record was 4 times).\n\nThanks a lot! I will add your Reported-by and Tested-by and send\nout the patches.","headers":{"Return-Path":"","X-Original-To":"patchwork-incoming@ozlabs.org","Delivered-To":"patchwork-incoming@ozlabs.org","Authentication-Results":["ozlabs.org;\n\tspf=none (mailfrom) smtp.mailfrom=vger.kernel.org\n\t(client-ip=209.132.180.67; helo=vger.kernel.org;\n\tenvelope-from=netdev-owner@vger.kernel.org;\n\treceiver=)","ozlabs.org; dkim=pass (2048-bit key;\n\tunprotected) header.d=gmail.com header.i=@gmail.com\n\theader.b=\"ErBtG0Yw\"; dkim-atps=neutral"],"Received":["from vger.kernel.org (vger.kernel.org [209.132.180.67])\n\tby ozlabs.org (Postfix) with ESMTP id 3yRvZw3pYCz9s7F\n\tfor ;\n\tThu, 2 Nov 2017 03:59:36 +1100 (AEDT)","(majordomo@vger.kernel.org) by vger.kernel.org via listexpand\n\tid S1755096AbdKAQ71 (ORCPT );\n\tWed, 1 Nov 2017 12:59:27 -0400","from mail-pg0-f53.google.com ([74.125.83.53]:55308 \"EHLO\n\tmail-pg0-f53.google.com\" rhost-flags-OK-OK-OK-OK) by vger.kernel.org\n\twith ESMTP id S1754844AbdKAQ7Z (ORCPT\n\t); Wed, 1 Nov 2017 12:59:25 -0400","by mail-pg0-f53.google.com with SMTP id 15so2581810pgc.12\n\tfor ; Wed, 01 Nov 2017 09:59:25 -0700 (PDT)","by 10.100.170.205 with HTTP; Wed, 1 Nov 2017 09:59:04 -0700 (PDT)"],"DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/relaxed;\n\td=gmail.com; s=20161025;\n\th=mime-version:in-reply-to:references:from:date:message-id:subject:to\n\t:cc; bh=Jo/2Nsv/cxGVl1CVgMGUC6LVELNUZW/gcXsdGlMzTjk=;\n\tb=ErBtG0Ywm0vfWj+JVOa6/UjNWWQHB7QTCbrry93S5Rpuq7ZSehubAoNwBtvgVeWhEC\n\t2CrWFrD3jiUpZP5ynNoP5Nza12cNDvOZvtoWttlqja8rdqurir+i36Qq9b0aAuRmwTKy\n\tws6J5b9Aw0j9SJScYMLyclvkmoQyteFf55iwaCVmjDeiM0FRoxtq9A13U2TwEH6Nkvag\n\tJZiYo8baAEgFLAnrW9Sta5sU50uJEn4z1INwZgiAgCKVPO0nkLHfKxuu9JK+XnEW55uR\n\taxKOhzBboQkMuvJ2W6bjU/uzCKILO2Mj3IcJMKG86goZ/TRCy5joNYBmVBXZ6T0YhW2K\n\t+aHA==","X-Google-DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/relaxed;\n\td=1e100.net; s=20161025;\n\th=x-gm-message-state:mime-version:in-reply-to:references:from:date\n\t:message-id:subject:to:cc;\n\tbh=Jo/2Nsv/cxGVl1CVgMGUC6LVELNUZW/gcXsdGlMzTjk=;\n\tb=fCk8cYkp4MnWrqrCqE5JsdihdSd8t4gPJr1lzEUQEErVdQEoSdBkSfV6OaVemJYW4N\n\tqPWNaTrWnVPPqYvhIza4dMDAhjK/XIiM3DemVP7ESeGXOW3PcT4h6/ZUvR8VMrgntbpe\n\tX6C0C1TDdrN2DoX9i2QJwEb9sUr3pfzFTJjJoK7IR87kPtzCl0KruW9ziILwzqob5Bye\n\tEQRomSYhax1tx+pIZTcax+/7XziZinnPkXn/+vtffAj1N5Jaq5CWG4NIm99xkRT1dWFA\n\tyuEh2C2jnCuJGRh5uQ93IAwQoIUyzZWnR2Pv1CtS0Bkgcw5OjecpkPRQCHLhFTV73jGm\n\tC+WQ==","X-Gm-Message-State":"AMCzsaXrHUWtJN0VYXeqg4pwNTU5JgQHJ5K51awnV4CLFO/pwHgpxSbn\n\t4ppRD4lsuHXSTHdc7qirJ1boArXqQIIUwc6MVF0=","X-Google-Smtp-Source":"ABhQp+ThhHF4CzrrjWfV33iFcmNU3nUS2aby7YlCKiBOxtGpa/BEhMlnqHIibna5soF9vXdnKECeZPaBiULXzqra/Qs=","X-Received":"by 10.99.119.9 with SMTP id s9mr554931pgc.54.1509555564882; Wed,\n\t01 Nov 2017 09:59:24 -0700 (PDT)","MIME-Version":"1.0","In-Reply-To":"","References":"<20171027012443.3306-1-xiyou.wangcong@gmail.com>\n\t\n\t\n\t\n\t\n\t<0dfff609-d67e-0701-399f-1dbe2ef09bf3@mojatatu.com>\n\t\n\t\n\t\n\t\n\t","From":"Cong Wang ","Date":"Wed, 1 Nov 2017 09:59:04 -0700","Message-ID":"","Subject":"Re: [Patch net 00/16] net_sched: fix races with RCU callbacks","To":"Lucas Bates ","Cc":"Jamal Hadi Salim ,\n\tLinux Kernel Network Developers ,\n\tChris Mi , Daniel Borkmann ,\n\tJiri Pirko , John Fastabend ,\n\t\"Paul E. McKenney\" ","Content-Type":"text/plain; charset=\"UTF-8\"","Sender":"netdev-owner@vger.kernel.org","Precedence":"bulk","List-ID":"","X-Mailing-List":"netdev@vger.kernel.org"}}]