| Message ID | 20090623150444.22490.27931.stgit@localhost |
|---|---|
| State | Changes Requested, archived |
| Delegated to: | David Miller |
| Headers | show |
Jesper Dangaard Brouer wrote: > I'm not sure which is are most optimal place to call rcu_barrier(). > The patch probably calls rcu_barrier() too much, but its a better > safe than sorry approach. > > There is embedded some comments that I would like Patrick McHardy > to look at. > > diff --git a/net/netfilter/nf_conntrack_core.c b/net/netfilter/nf_conntrack_core.c > index 5f72b94..cea4537 100644 > --- a/net/netfilter/nf_conntrack_core.c > +++ b/net/netfilter/nf_conntrack_core.c > @@ -1084,6 +1084,8 @@ static void nf_conntrack_cleanup_init_net(void) > { > nf_conntrack_helper_fini(); > nf_conntrack_proto_fini(); > + rcu_barrier(); > + /* Need to wait for call_rcu() before dealloc the kmem_cache */ > kmem_cache_destroy(nf_conntrack_cachep); Which call_rcu() is this referring to? If its the conntrack destruction, that one is gone in the current kernel and I think destruction is handled properly by the sl*b-allocators (SLAB_DESTROY_BY_RCU). > @@ -1118,6 +1120,9 @@ void nf_conntrack_cleanup(struct net *net) > /* This makes sure all current packets have passed through > netfilter framework. Roll on, two-stage module > delete... */ > + /* hawk@comx.dk 2009-06-20: Think this should be replaced by a > + rcu_barrier() ??? > + */ > synchronize_net(); AFAICT this one is used to make sure the old value of the ip_ct_attach hook is not visible anymore before beginning cleanup and is not needed for anything else. > nf_conntrack_cleanup_net(net); > diff --git a/net/netfilter/nf_conntrack_standalone.c b/net/netfilter/nf_conntrack_standalone.c > index 1935153..29c6cd0 100644 > --- a/net/netfilter/nf_conntrack_standalone.c > +++ b/net/netfilter/nf_conntrack_standalone.c > @@ -500,6 +500,8 @@ static void nf_conntrack_net_exit(struct net *net) > nf_conntrack_standalone_fini_sysctl(net); > nf_conntrack_standalone_fini_proc(net); > nf_conntrack_cleanup(net); > + /* hawk@comx.dk: Think rcu_barrier() should to be called earlier? */ > + rcu_barrier(); /* Wait for completion of call_rcu()'s */ > } Which call_rcu() is this referring to? We should place them in the cleanup sub-functions to make this clearly visible. -- To unsubscribe from this list: send the line "unsubscribe netdev" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
On Tue, 2009-06-23 at 18:23 +0200, Patrick McHardy wrote: > Jesper Dangaard Brouer wrote: > > I'm not sure which is are most optimal place to call rcu_barrier(). > > The patch probably calls rcu_barrier() too much, but its a better > > safe than sorry approach. > > > > There is embedded some comments that I would like Patrick McHardy > > to look at. > > > > diff --git a/net/netfilter/nf_conntrack_core.c b/net/netfilter/nf_conntrack_core.c > > index 5f72b94..cea4537 100644 > > --- a/net/netfilter/nf_conntrack_core.c > > +++ b/net/netfilter/nf_conntrack_core.c > > @@ -1084,6 +1084,8 @@ static void nf_conntrack_cleanup_init_net(void) > > { > > nf_conntrack_helper_fini(); > > nf_conntrack_proto_fini(); > > + rcu_barrier(); > > + /* Need to wait for call_rcu() before dealloc the kmem_cache */ > > kmem_cache_destroy(nf_conntrack_cachep); > > Which call_rcu() is this referring to? It is the call_rcu() in nf_conntrack_expect.c (which is linked into nf_conntrack.ko). But that also means that it should have been the slab cache called "nf_ct_expect_cachep" we should have waited for... (and I also notice that "nf_ct_expect_cachep" is missing the SLAB_DESTROY_BY_RCU flag, and the SLAB_DESTROY_BY_RCU flag should be removed from "nf_conntrack_cachep") > If its the conntrack destruction, > that one is gone in the current kernel and I think destruction is > handled properly by the sl*b-allocators (SLAB_DESTROY_BY_RCU). Just dived into the slab.c code and noticed that it also is flawed, ARGH!. When the SLAB_DESTROY_BY_RCU flags is set, it only calls synchronize_rcu() and not rcu_barrier() as it should! I'll fix that up in another patch series... Looking into slub and slob at the moment, and it seems that they schedule another call_rcu callback for freeing when the SLAB_DESTROY_BY_RCU flags is set. That seems racy to me... Paul? > > @@ -1118,6 +1120,9 @@ void nf_conntrack_cleanup(struct net *net) > > /* This makes sure all current packets have passed through > > netfilter framework. Roll on, two-stage module > > delete... */ > > + /* hawk@comx.dk 2009-06-20: Think this should be replaced by a > > + rcu_barrier() ??? > > + */ > > synchronize_net(); > > AFAICT this one is used to make sure the old value of the ip_ct_attach > hook is not visible anymore before beginning cleanup and is not needed > for anything else. Fine! > > nf_conntrack_cleanup_net(net); > > diff --git a/net/netfilter/nf_conntrack_standalone.c b/net/netfilter/nf_conntrack_standalone.c > > index 1935153..29c6cd0 100644 > > --- a/net/netfilter/nf_conntrack_standalone.c > > +++ b/net/netfilter/nf_conntrack_standalone.c > > @@ -500,6 +500,8 @@ static void nf_conntrack_net_exit(struct net *net) > > nf_conntrack_standalone_fini_sysctl(net); > > nf_conntrack_standalone_fini_proc(net); > > nf_conntrack_cleanup(net); > > + /* hawk@comx.dk: Think rcu_barrier() should to be called earlier? */ > > + rcu_barrier(); /* Wait for completion of call_rcu()'s */ > > } > > Which call_rcu() is this referring to? We should place them in > the cleanup sub-functions to make this clearly visible. This call_rcu() is the one done in nf_conntrack_extend.c:114 (notice "_extend" NOT "_expect"), which calls __nf_ct_ext_free_rcu(). Guess this rcu_barrier() should then be move to nf_ct_extend_unregister() right? (it already invokes a synchronize_rcu() that should be replaced by rcu_barrier()). Although this means the nf_ct_extend_unregister() will be called three times in nf_conntrack_cleanup_net() when unregistering ecache, acct and expect. Thank you for your feedback :-) ... I'll post a new v2 patch...
diff --git a/net/netfilter/nf_conntrack_core.c b/net/netfilter/nf_conntrack_core.c index 5f72b94..cea4537 100644 --- a/net/netfilter/nf_conntrack_core.c +++ b/net/netfilter/nf_conntrack_core.c @@ -1084,6 +1084,8 @@ static void nf_conntrack_cleanup_init_net(void) { nf_conntrack_helper_fini(); nf_conntrack_proto_fini(); + rcu_barrier(); + /* Need to wait for call_rcu() before dealloc the kmem_cache */ kmem_cache_destroy(nf_conntrack_cachep); } @@ -1118,6 +1120,9 @@ void nf_conntrack_cleanup(struct net *net) /* This makes sure all current packets have passed through netfilter framework. Roll on, two-stage module delete... */ + /* hawk@comx.dk 2009-06-20: Think this should be replaced by a + rcu_barrier() ??? + */ synchronize_net(); nf_conntrack_cleanup_net(net); diff --git a/net/netfilter/nf_conntrack_standalone.c b/net/netfilter/nf_conntrack_standalone.c index 1935153..29c6cd0 100644 --- a/net/netfilter/nf_conntrack_standalone.c +++ b/net/netfilter/nf_conntrack_standalone.c @@ -500,6 +500,8 @@ static void nf_conntrack_net_exit(struct net *net) nf_conntrack_standalone_fini_sysctl(net); nf_conntrack_standalone_fini_proc(net); nf_conntrack_cleanup(net); + /* hawk@comx.dk: Think rcu_barrier() should to be called earlier? */ + rcu_barrier(); /* Wait for completion of call_rcu()'s */ } static struct pernet_operations nf_conntrack_net_ops = {
I'm not sure which is are most optimal place to call rcu_barrier(). The patch probably calls rcu_barrier() too much, but its a better safe than sorry approach. There is embedded some comments that I would like Patrick McHardy to look at. Signed-off-by: Jesper Dangaard Brouer <hawk@comx.dk> --- net/netfilter/nf_conntrack_core.c | 5 +++++ net/netfilter/nf_conntrack_standalone.c | 2 ++ 2 files changed, 7 insertions(+), 0 deletions(-) -- To unsubscribe from this list: send the line "unsubscribe netdev" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html