From patchwork Sun Feb 25 19:47:19 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Ahern X-Patchwork-Id: 877610 X-Patchwork-Delegate: davem@davemloft.net Return-Path: X-Original-To: patchwork-incoming@ozlabs.org Delivered-To: patchwork-incoming@ozlabs.org Authentication-Results: ozlabs.org; spf=none (mailfrom) smtp.mailfrom=vger.kernel.org (client-ip=209.132.180.67; helo=vger.kernel.org; envelope-from=netdev-owner@vger.kernel.org; receiver=) Authentication-Results: ozlabs.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.b="d+wSRD8e"; dkim-atps=neutral Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by ozlabs.org (Postfix) with ESMTP id 3zqFqt0fgcz9s2S for ; Mon, 26 Feb 2018 06:48:10 +1100 (AEDT) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1752055AbeBYTsH (ORCPT ); Sun, 25 Feb 2018 14:48:07 -0500 Received: from mail-pg0-f66.google.com ([74.125.83.66]:40953 "EHLO mail-pg0-f66.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1752001AbeBYTrx (ORCPT ); Sun, 25 Feb 2018 14:47:53 -0500 Received: by mail-pg0-f66.google.com with SMTP id g2so5380547pgn.7 for ; Sun, 25 Feb 2018 11:47:52 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=mJaE12+vzEzKIC1P2CZrstRIChjnbrW/pwUQQ02QIkU=; b=d+wSRD8eU3M3RjNz5vEaE4aqRnl1tNrky5CpWuAPxpiKGFSpUiRPighHkpPG9a/CfS 18U6/5ZEkh1ZvyaBhDr6kO/O5Wm5GIaQGk053qcXMJ50RcKW2r4eFB9OmxViaO/F290M IqkApevN/+dlL1AWkvfaQDKBc7fFrfSsJJziYmrqsluBw5DyPoIzb+O+psMCQjfyg8ZS z423G5PrXmIJrs+6YohNsotC+PLsCtf6QcYKRq8tEupnwZXTjfgY6LS91QUcPIPy2sYg vHJt7D7/d6v1U6PpNQJHjSm/NpMe6Dnt/X+zExouJh3Rsyu/4LTlnxRMl+dlafWdjjBl oulg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=mJaE12+vzEzKIC1P2CZrstRIChjnbrW/pwUQQ02QIkU=; b=fb2R2OQDKAf7X09ERbdZ1x8ACIeB5LyE19hASLy19dGDFS4dbphiybxhqkjHXnFYkR Lqw7MEU27ogaYZZc0t9vt0iZ5VxAbmqP5UpARi9CeT/206ERFAemP6G3AWExtd2xGYGy xfSXp2GMszfM6Sp0+UOONYHj2sBaEZc9gBy9lAtKNkG3SZXbO1SuQYpK9VdgCwDYPqnn EMkk693vp+sAskdDwRLVwdZOslXe6Fh2Bs8VWRjcTDLT5gESbtiNhXnofVigk4qbFuQC otuS530cegOAL+GNE6PAuCDWlp3Y/BwvWC83s0xNmquwENRtRCpxkp1O4Zqs5f/P6lDR 3viQ== X-Gm-Message-State: APf1xPCYTm5LK1rzK84HllBWqt0tVYHwwhE0WtNtcrCfsJPiNYvNGDBg wtzrgRRoXKXbVbTunywBH4+Q7g== X-Google-Smtp-Source: AH8x2263U2f8qTKVKvpxe477qVKiT5EGgEzzdT/Me86Hiw9FjVpXnODS1TxXCQjim0FF/qRfiGGxVA== X-Received: by 10.101.80.3 with SMTP id f3mr6734127pgo.242.1519588071629; Sun, 25 Feb 2018 11:47:51 -0800 (PST) Received: from kenny.it.cumulusnetworks.com. (fw.cumulusnetworks.com. [216.129.126.126]) by smtp.googlemail.com with ESMTPSA id z4sm12414196pgb.4.2018.02.25.11.47.50 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Sun, 25 Feb 2018 11:47:51 -0800 (PST) From: David Ahern To: netdev@vger.kernel.org Cc: davem@davemloft.net, idosch@idosch.org, roopa@cumulusnetworks.com, eric.dumazet@gmail.com, weiwan@google.com, kafai@fb.com, yoshfuji@linux-ipv6.org, David Ahern Subject: [PATCH RFC net-next 09/20] net/ipv6: move metrics from dst to rt6_info Date: Sun, 25 Feb 2018 11:47:19 -0800 Message-Id: <20180225194730.30063-10-dsahern@gmail.com> X-Mailer: git-send-email 2.11.0 In-Reply-To: <20180225194730.30063-1-dsahern@gmail.com> References: <20180225194730.30063-1-dsahern@gmail.com> Sender: netdev-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org Similar to IPv4, add fib metrics to the fib struct, which at the moment is rt6_info. Will be moved to fib6_info in a later patch. Copy metrics into dst by reference using refcount. Signed-off-by: David Ahern --- include/net/ip6_fib.h | 13 +-- net/core/dst.c | 1 + net/ipv6/ip6_fib.c | 49 +-------- net/ipv6/ndisc.c | 5 +- net/ipv6/route.c | 267 +++++++++++++++++++------------------------------- 5 files changed, 115 insertions(+), 220 deletions(-) diff --git a/include/net/ip6_fib.h b/include/net/ip6_fib.h index 7678ae3de44a..da81669b9c90 100644 --- a/include/net/ip6_fib.h +++ b/include/net/ip6_fib.h @@ -94,11 +94,6 @@ struct fib6_gc_args { #define FIB6_SUBTREE(fn) (rcu_dereference_protected((fn)->subtree, 1)) #endif -struct mx6_config { - const u32 *mx; - DECLARE_BITMAP(mx_valid, RTAX_MAX); -}; - /* * routing information * @@ -176,7 +171,6 @@ struct rt6_info { struct rt6_exception_bucket __rcu *rt6i_exception_bucket; u32 rt6i_metric; - u32 rt6i_pmtu; /* more non-fragment space at head required */ unsigned short rt6i_nfheader_len; u8 rt6i_protocol; @@ -185,6 +179,10 @@ struct rt6_info { should_flush:1, unused:6; + struct dst_metrics *fib6_metrics; +#define fib6_pmtu fib6_metrics->metrics[RTAX_MTU-1] +#define fib6_hoplimit fib6_metrics->metrics[RTAX_HOPLIMIT-1] +#define fib6_metric_lock fib6_metrics->metrics[RTAX_LOCK - 1] struct fib6_nh fib6_nh; }; @@ -388,8 +386,7 @@ void fib6_clean_all(struct net *net, int (*func)(struct rt6_info *, void *arg), void *arg); int fib6_add(struct fib6_node *root, struct rt6_info *rt, - struct nl_info *info, struct mx6_config *mxc, - struct netlink_ext_ack *extack); + struct nl_info *info, struct netlink_ext_ack *extack); int fib6_del(struct rt6_info *rt, struct nl_info *info); void inet6_rt_notify(int event, struct rt6_info *rt, struct nl_info *info, diff --git a/net/core/dst.c b/net/core/dst.c index 5f70bc832bec..2c23eadd6f7e 100644 --- a/net/core/dst.c +++ b/net/core/dst.c @@ -59,6 +59,7 @@ const struct dst_metrics dst_default_metrics = { */ .refcnt = REFCOUNT_INIT(1), }; +EXPORT_SYMBOL(dst_default_metrics); void dst_init(struct dst_entry *dst, struct dst_ops *ops, struct net_device *dev, int initial_ref, int initial_obsolete, diff --git a/net/ipv6/ip6_fib.c b/net/ipv6/ip6_fib.c index 134d86483f3c..faa2b46349df 100644 --- a/net/ipv6/ip6_fib.c +++ b/net/ipv6/ip6_fib.c @@ -800,38 +800,6 @@ static struct fib6_node *fib6_add_1(struct net *net, return ln; } -static void fib6_copy_metrics(u32 *mp, const struct mx6_config *mxc) -{ - int i; - - for (i = 0; i < RTAX_MAX; i++) { - if (test_bit(i, mxc->mx_valid)) - mp[i] = mxc->mx[i]; - } -} - -static int fib6_commit_metrics(struct dst_entry *dst, struct mx6_config *mxc) -{ - if (!mxc->mx) - return 0; - - if (dst->flags & DST_HOST) { - u32 *mp = dst_metrics_write_ptr(dst); - - if (unlikely(!mp)) - return -ENOMEM; - - fib6_copy_metrics(mp, mxc); - } else { - dst_init_metrics(dst, mxc->mx, false); - - /* We've stolen mx now. */ - mxc->mx = NULL; - } - - return 0; -} - static void fib6_purge_rt(struct rt6_info *rt, struct fib6_node *fn, struct net *net) { @@ -865,7 +833,7 @@ static void fib6_purge_rt(struct rt6_info *rt, struct fib6_node *fn, */ static int fib6_add_rt2node(struct fib6_node *fn, struct rt6_info *rt, - struct nl_info *info, struct mx6_config *mxc, + struct nl_info *info, struct netlink_ext_ack *extack) { struct rt6_info *leaf = rcu_dereference_protected(fn->leaf, @@ -880,7 +848,6 @@ static int fib6_add_rt2node(struct fib6_node *fn, struct rt6_info *rt, int found = 0; bool rt_can_ecmp = rt6_qualify_for_ecmp(rt); u16 nlflags = NLM_F_EXCL; - int err; if (info->nlh && (info->nlh->nlmsg_flags & NLM_F_APPEND)) nlflags |= NLM_F_APPEND; @@ -922,7 +889,7 @@ static int fib6_add_rt2node(struct fib6_node *fn, struct rt6_info *rt, rt6_clean_expires(iter); else rt6_set_expires(iter, rt->dst.expires); - iter->rt6i_pmtu = rt->rt6i_pmtu; + iter->fib6_pmtu = rt->fib6_pmtu; return -EEXIST; } /* If we have the same destination and the same metric, @@ -1001,9 +968,6 @@ static int fib6_add_rt2node(struct fib6_node *fn, struct rt6_info *rt, add: nlflags |= NLM_F_CREATE; - err = fib6_commit_metrics(&rt->dst, mxc); - if (err) - return err; rcu_assign_pointer(rt->rt6_next, iter); atomic_inc(&rt->rt6i_ref); @@ -1030,10 +994,6 @@ static int fib6_add_rt2node(struct fib6_node *fn, struct rt6_info *rt, return -ENOENT; } - err = fib6_commit_metrics(&rt->dst, mxc); - if (err) - return err; - atomic_inc(&rt->rt6i_ref); rcu_assign_pointer(rt->rt6i_node, fn); rt->rt6_next = iter->rt6_next; @@ -1126,8 +1086,7 @@ void fib6_update_sernum_upto_root(struct net *net, struct rt6_info *rt) */ int fib6_add(struct fib6_node *root, struct rt6_info *rt, - struct nl_info *info, struct mx6_config *mxc, - struct netlink_ext_ack *extack) + struct nl_info *info, struct netlink_ext_ack *extack) { struct fib6_table *table = rt->rt6i_table; struct fib6_node *fn, *pn = NULL; @@ -1235,7 +1194,7 @@ int fib6_add(struct fib6_node *root, struct rt6_info *rt, } #endif - err = fib6_add_rt2node(fn, rt, info, mxc, extack); + err = fib6_add_rt2node(fn, rt, info, extack); if (!err) { __fib6_update_sernum_upto_root(rt, sernum); fib6_start_gc(info->nl_net, rt); diff --git a/net/ipv6/ndisc.c b/net/ipv6/ndisc.c index 3b827475fb0d..33f9e3dc526a 100644 --- a/net/ipv6/ndisc.c +++ b/net/ipv6/ndisc.c @@ -1324,8 +1324,7 @@ static void ndisc_router_discovery(struct sk_buff *skb) if (in6_dev->cnf.accept_ra_min_hop_limit <= ra_msg->icmph.icmp6_hop_limit) { in6_dev->cnf.hop_limit = ra_msg->icmph.icmp6_hop_limit; if (rt) - dst_metric_set(&rt->dst, RTAX_HOPLIMIT, - ra_msg->icmph.icmp6_hop_limit); + rt->fib6_hoplimit = ra_msg->icmph.icmp6_hop_limit; } else { ND_PRINTK(2, warn, "RA: Got route advertisement with lower hop_limit than minimum\n"); } @@ -1479,7 +1478,7 @@ static void ndisc_router_discovery(struct sk_buff *skb) in6_dev->cnf.mtu6 = mtu; if (rt) - dst_metric_set(&rt->dst, RTAX_MTU, mtu); + rt->fib6_pmtu = mtu; rt6_mtu_change(skb->dev, mtu); } diff --git a/net/ipv6/route.c b/net/ipv6/route.c index b56f56508970..1649e78b019d 100644 --- a/net/ipv6/route.c +++ b/net/ipv6/route.c @@ -96,12 +96,11 @@ static void ip6_rt_update_pmtu(struct dst_entry *dst, struct sock *sk, struct sk_buff *skb, u32 mtu); static void rt6_do_redirect(struct dst_entry *dst, struct sock *sk, struct sk_buff *skb); -static void rt6_dst_from_metrics_check(struct rt6_info *rt); static int rt6_score_route(struct rt6_info *rt, int oif, int strict); static size_t rt6_nlmsg_size(struct rt6_info *rt); -static int rt6_fill_node(struct net *net, - struct sk_buff *skb, struct rt6_info *rt, - struct in6_addr *dst, struct in6_addr *src, +static int rt6_fill_node(struct net *net, struct sk_buff *skb, + struct rt6_info *rt, struct dst_entry *dst, + struct in6_addr *dest, struct in6_addr *src, int iif, int type, u32 portid, u32 seq, unsigned int flags); static struct rt6_info *rt6_find_cached_rt(struct rt6_info *rt, @@ -183,23 +182,6 @@ static void rt6_uncached_list_flush_dev(struct net *net, struct net_device *dev) } } -static u32 *rt6_pcpu_cow_metrics(struct rt6_info *rt) -{ - return dst_metrics_write_ptr(&rt->from->dst); -} - -static u32 *ipv6_cow_metrics(struct dst_entry *dst, unsigned long old) -{ - struct rt6_info *rt = (struct rt6_info *)dst; - - if (rt->rt6i_flags & RTF_PCPU) - return rt6_pcpu_cow_metrics(rt); - else if (rt->rt6i_flags & RTF_CACHE) - return NULL; - else - return dst_cow_metrics_generic(dst, old); -} - static inline const void *choose_neigh_daddr(struct rt6_info *rt, struct sk_buff *skb, const void *daddr) @@ -249,7 +231,7 @@ static struct dst_ops ip6_dst_ops_template = { .check = ip6_dst_check, .default_advmss = ip6_default_advmss, .mtu = ip6_mtu, - .cow_metrics = ipv6_cow_metrics, + .cow_metrics = dst_cow_metrics_generic, .destroy = ip6_dst_destroy, .ifdown = ip6_dst_ifdown, .negative_advice = ip6_negative_advice, @@ -353,6 +335,7 @@ static void rt6_info_init(struct rt6_info *rt) memset(dst + 1, 0, sizeof(*rt) - sizeof(*dst)); INIT_LIST_HEAD(&rt->rt6i_siblings); INIT_LIST_HEAD(&rt->rt6i_uncached); + rt->fib6_metrics = (struct dst_metrics *)&dst_default_metrics; } /* allocate dst with ip6_dst_ops */ @@ -395,6 +378,7 @@ static void ip6_dst_destroy(struct dst_entry *dst) struct rt6_exception_bucket *bucket; struct rt6_info *from = rt->from; struct inet6_dev *idev; + struct dst_metrics *m; dst_destroy_metrics_generic(dst); free_percpu(rt->rt6i_pcpu); @@ -411,6 +395,10 @@ static void ip6_dst_destroy(struct dst_entry *dst) kfree(bucket); } + m = rt->fib6_metrics; + if (m != &dst_default_metrics && refcount_dec_and_test(&m->refcnt)) + kfree(m); + rt->from = NULL; dst_release(&from->dst); } @@ -994,7 +982,11 @@ static void rt6_set_from(struct rt6_info *rt, struct rt6_info *from) rt->rt6i_flags &= ~RTF_EXPIRES; dst_hold(&from->dst); rt->from = from; - dst_init_metrics(&rt->dst, dst_metrics_ptr(&from->dst), true); + dst_init_metrics(&rt->dst, from->fib6_metrics->metrics, true); + if (from->fib6_metrics != &dst_default_metrics) { + rt->dst._metrics |= DST_METRICS_REFCOUNTED; + refcount_inc(&from->fib6_metrics->refcnt); + } } static void ip6_rt_copy_init(struct rt6_info *rt, struct rt6_info *ort) @@ -1132,7 +1124,6 @@ EXPORT_SYMBOL(rt6_lookup); */ static int __ip6_ins_rt(struct rt6_info *rt, struct nl_info *info, - struct mx6_config *mxc, struct netlink_ext_ack *extack) { int err; @@ -1140,7 +1131,7 @@ static int __ip6_ins_rt(struct rt6_info *rt, struct nl_info *info, table = rt->rt6i_table; spin_lock_bh(&table->tb6_lock); - err = fib6_add(&table->tb6_root, rt, info, mxc, extack); + err = fib6_add(&table->tb6_root, rt, info, extack); spin_unlock_bh(&table->tb6_lock); return err; @@ -1149,11 +1140,10 @@ static int __ip6_ins_rt(struct rt6_info *rt, struct nl_info *info, int ip6_ins_rt(struct net *net, struct rt6_info *rt) { struct nl_info info = { .nl_net = net, }; - struct mx6_config mxc = { .mx = NULL, }; /* Hold dst to account for the reference from the fib6 tree */ dst_hold(&rt->dst); - return __ip6_ins_rt(rt, &info, &mxc, NULL); + return __ip6_ins_rt(rt, &info, NULL); } static struct rt6_info *ip6_rt_cache_alloc(struct rt6_info *ort, @@ -1224,8 +1214,8 @@ static struct rt6_info *rt6_get_pcpu_route(struct rt6_info *rt) p = this_cpu_ptr(rt->rt6i_pcpu); pcpu_rt = *p; - if (pcpu_rt && ip6_hold_safe(NULL, &pcpu_rt, false)) - rt6_dst_from_metrics_check(pcpu_rt); + if (pcpu_rt) + ip6_hold_safe(NULL, &pcpu_rt, false); return pcpu_rt; } @@ -1246,7 +1236,6 @@ static struct rt6_info *rt6_make_pcpu_route(struct net *net, prev = cmpxchg(p, NULL, pcpu_rt); BUG_ON(prev); - rt6_dst_from_metrics_check(pcpu_rt); return pcpu_rt; } @@ -1376,6 +1365,16 @@ __rt6_find_exception_rcu(struct rt6_exception_bucket **bucket, return NULL; } +static unsigned int fib6_mtu(const struct rt6_info *rt) +{ + unsigned int mtu; + + mtu = rt->fib6_pmtu ? : rt->rt6i_idev->cnf.mtu6; + mtu = min_t(unsigned int, mtu, IP6_MAX_MTU); + + return mtu - lwtunnel_headroom(rt->fib6_nh.nh_lwtstate, mtu); +} + static int rt6_insert_exception(struct rt6_info *nrt, struct rt6_info *ort) { @@ -1428,7 +1427,7 @@ static int rt6_insert_exception(struct rt6_info *nrt, * Only insert this exception route if its mtu * is less than ort's mtu value. */ - if (nrt->rt6i_pmtu >= dst_mtu(&ort->dst)) { + if (dst_metric_raw(&nrt->dst, RTAX_MTU) >= fib6_mtu(ort)) { err = -EINVAL; goto out; } @@ -1638,13 +1637,15 @@ static void rt6_exceptions_update_pmtu(struct rt6_info *rt, int mtu) for (i = 0; i < FIB6_EXCEPTION_BUCKET_SIZE; i++) { hlist_for_each_entry(rt6_ex, &bucket->chain, hlist) { struct rt6_info *entry = rt6_ex->rt6i; + u32 dst_mtu = dst_metric(&entry->dst, RTAX_MTU); + /* For RTF_CACHE with rt6i_pmtu == 0 * (i.e. a redirected route), - * the metrics of its rt->dst.from has already + * the metrics of its rt->from has already * been updated. */ - if (entry->rt6i_pmtu && entry->rt6i_pmtu > mtu) - entry->rt6i_pmtu = mtu; + if (dst_mtu && dst_mtu > mtu) + dst_metric_set(&entry->dst, RTAX_MTU, mtu); } bucket++; } @@ -1809,10 +1810,9 @@ struct rt6_info *ip6_pol_route(struct net *net, struct fib6_table *table, trace_fib6_table_lookup(net, rt, table, fl6); return rt; } else if (rt->rt6i_flags & RTF_CACHE) { - if (ip6_hold_safe(net, &rt, true)) { + if (ip6_hold_safe(net, &rt, true)) dst_use_noref(&rt->dst, jiffies); - rt6_dst_from_metrics_check(rt); - } + rcu_read_unlock(); trace_fib6_table_lookup(net, rt, table, fl6); return rt; @@ -2046,13 +2046,6 @@ struct dst_entry *ip6_blackhole_route(struct net *net, struct dst_entry *dst_ori * Destination cache support functions */ -static void rt6_dst_from_metrics_check(struct rt6_info *rt) -{ - if (rt->from && - dst_metrics_ptr(&rt->dst) != dst_metrics_ptr(&rt->from->dst)) - dst_init_metrics(&rt->dst, dst_metrics_ptr(&rt->from->dst), true); -} - static struct dst_entry *rt6_check(struct rt6_info *rt, u32 cookie) { u32 rt_cookie = 0; @@ -2087,8 +2080,6 @@ static struct dst_entry *ip6_dst_check(struct dst_entry *dst, u32 cookie) * into this function always. */ - rt6_dst_from_metrics_check(rt); - if (rt->rt6i_flags & RTF_PCPU || (unlikely(!list_empty(&rt->rt6i_uncached)) && rt->from)) return rt6_dst_from_check(rt, cookie); @@ -2141,8 +2132,8 @@ static void rt6_do_update_pmtu(struct rt6_info *rt, u32 mtu) { struct net *net = dev_net(rt->dst.dev); + dst_metric_set(&rt->dst, RTAX_MTU, mtu); rt->rt6i_flags |= RTF_MODIFIED; - rt->rt6i_pmtu = mtu; rt6_update_expires(rt, net->ipv6.sysctl.ip6_rt_mtu_expires); } @@ -2188,10 +2179,10 @@ static void __ip6_rt_update_pmtu(struct dst_entry *dst, const struct sock *sk, } else if (daddr) { struct rt6_info *nrt6; - nrt6 = ip6_rt_cache_alloc(rt6, daddr, saddr); + nrt6 = ip6_rt_cache_alloc(rt6->from, daddr, saddr); if (nrt6) { rt6_do_update_pmtu(nrt6, mtu); - if (rt6_insert_exception(nrt6, rt6)) + if (rt6_insert_exception(nrt6, rt6->from)) dst_release_immediate(&nrt6->dst); } } @@ -2413,12 +2404,8 @@ static unsigned int ip6_default_advmss(const struct dst_entry *dst) static unsigned int ip6_mtu(const struct dst_entry *dst) { - const struct rt6_info *rt = (const struct rt6_info *)dst; - unsigned int mtu = rt->rt6i_pmtu; struct inet6_dev *idev; - - if (mtu) - goto out; + unsigned int mtu; mtu = dst_metric_raw(dst, RTAX_MTU); if (mtu) @@ -2502,60 +2489,24 @@ static int ip6_dst_gc(struct dst_ops *ops) return entries > rt_max_size; } -static int ip6_convert_metrics(struct mx6_config *mxc, - const struct fib6_config *cfg) +static int ip6_convert_metrics(struct net *net, struct rt6_info *rt, + struct fib6_config *cfg) { - struct net *net = cfg->fc_nlinfo.nl_net; - bool ecn_ca = false; - struct nlattr *nla; - int remaining; - u32 *mp; - - if (!cfg->fc_mx) - return 0; - - mp = kzalloc(sizeof(u32) * RTAX_MAX, GFP_KERNEL); - if (unlikely(!mp)) - return -ENOMEM; - - nla_for_each_attr(nla, cfg->fc_mx, cfg->fc_mx_len, remaining) { - int type = nla_type(nla); - u32 val; - - if (!type) - continue; - if (unlikely(type > RTAX_MAX)) - goto err; - - if (type == RTAX_CC_ALGO) { - char tmp[TCP_CA_NAME_MAX]; + int err = 0; - nla_strlcpy(tmp, nla, sizeof(tmp)); - val = tcp_ca_get_key_by_name(net, tmp, &ecn_ca); - if (val == TCP_CA_UNSPEC) - goto err; - } else { - val = nla_get_u32(nla); - } - if (type == RTAX_HOPLIMIT && val > 255) - val = 255; - if (type == RTAX_FEATURES && (val & ~RTAX_FEATURE_MASK)) - goto err; + if (cfg->fc_mx) { + rt->fib6_metrics = kzalloc(sizeof(*rt->fib6_metrics), + GFP_KERNEL); + if (unlikely(!rt->fib6_metrics)) + return -ENOMEM; - mp[type - 1] = val; - __set_bit(type - 1, mxc->mx_valid); - } + refcount_set(&rt->fib6_metrics->refcnt, 1); - if (ecn_ca) { - __set_bit(RTAX_FEATURES - 1, mxc->mx_valid); - mp[RTAX_FEATURES - 1] |= DST_FEATURE_ECN_CA; + err = dst_metrics_convert(net, cfg->fc_mx, cfg->fc_mx_len, + rt->fib6_metrics->metrics); } - mxc->mx = mp; - return 0; - err: - kfree(mp); - return -EINVAL; + return err; } static struct rt6_info *ip6_nh_lookup_table(struct net *net, @@ -2762,6 +2713,10 @@ static struct rt6_info *ip6_route_info_create(struct fib6_config *cfg, goto out; } + err = ip6_convert_metrics(net, rt, cfg); + if (err < 0) + goto out; + if (cfg->fc_flags & RTF_EXPIRES) rt6_set_expires(rt, jiffies + clock_t_to_jiffies(cfg->fc_expires)); @@ -2931,32 +2886,16 @@ static struct rt6_info *ip6_route_info_create(struct fib6_config *cfg, return ERR_PTR(err); } -int ip6_route_add(struct fib6_config *cfg, - struct netlink_ext_ack *extack) +int ip6_route_add(struct fib6_config *cfg, struct netlink_ext_ack *extack) { - struct mx6_config mxc = { .mx = NULL, }; struct rt6_info *rt; int err; rt = ip6_route_info_create(cfg, extack); - if (IS_ERR(rt)) { - err = PTR_ERR(rt); - rt = NULL; - goto out; - } + if (IS_ERR(rt)) + return PTR_ERR(rt); - err = ip6_convert_metrics(&mxc, cfg); - if (err) - goto out; - - err = __ip6_ins_rt(rt, &cfg->fc_nlinfo, &mxc, extack); - - kfree(mxc.mx); - - return err; -out: - if (rt) - dst_release_immediate(&rt->dst); + err = __ip6_ins_rt(rt, &cfg->fc_nlinfo, extack); return err; } @@ -3010,7 +2949,7 @@ static int __ip6_del_rt_siblings(struct rt6_info *rt, struct fib6_config *cfg) if (skb) { u32 seq = info->nlh ? info->nlh->nlmsg_seq : 0; - if (rt6_fill_node(net, skb, rt, + if (rt6_fill_node(net, skb, rt, NULL, NULL, NULL, 0, RTM_DELROUTE, info->portid, seq, 0) < 0) { kfree_skb(skb); @@ -3201,7 +3140,7 @@ static void rt6_do_redirect(struct dst_entry *dst, struct sock *sk, struct sk_bu * a cached route because rt6_insert_exception() will * takes care of it */ - if (rt6_insert_exception(nrt, rt)) { + if (rt6_insert_exception(nrt, rt->from)) { dst_release_immediate(&nrt->dst); goto out; } @@ -3878,15 +3817,15 @@ static int rt6_mtu_change_route(struct rt6_info *rt, void *p_arg) also have the lowest MTU, TOO BIG MESSAGE will be lead to PMTU discovery. */ - if (rt->fib6_nh.nh_dev == arg->dev && - dst_metric_raw(&rt->dst, RTAX_MTU) && - !dst_metric_locked(&rt->dst, RTAX_MTU)) { + if (rt->fib6_nh.nh_dev == arg->dev && rt->fib6_pmtu && + !(rt->fib6_metric_lock & (1 << RTAX_MTU))) { + u32 mtu = rt->fib6_pmtu; + + if (mtu >= arg->mtu || + (mtu < arg->mtu && mtu == idev->cnf.mtu6)) + rt->fib6_pmtu = arg->mtu; + spin_lock_bh(&rt6_exception_lock); - if (dst_mtu(&rt->dst) >= arg->mtu || - (dst_mtu(&rt->dst) < arg->mtu && - dst_mtu(&rt->dst) == idev->cnf.mtu6)) { - dst_metric_set(&rt->dst, RTAX_MTU, arg->mtu); - } rt6_exceptions_update_pmtu(rt, arg->mtu); spin_unlock_bh(&rt6_exception_lock); } @@ -4047,7 +3986,6 @@ static int rtm_to_fib6_config(struct sk_buff *skb, struct nlmsghdr *nlh, struct rt6_nh { struct rt6_info *rt6_info; struct fib6_config r_cfg; - struct mx6_config mxc; struct list_head next; }; @@ -4062,7 +4000,8 @@ static void ip6_print_replace_route_err(struct list_head *rt6_nh_list) } } -static int ip6_route_info_append(struct list_head *rt6_nh_list, +static int ip6_route_info_append(struct net *net, + struct list_head *rt6_nh_list, struct rt6_info *rt, struct fib6_config *r_cfg) { struct rt6_nh *nh; @@ -4078,7 +4017,7 @@ static int ip6_route_info_append(struct list_head *rt6_nh_list, if (!nh) return -ENOMEM; nh->rt6_info = rt; - err = ip6_convert_metrics(&nh->mxc, r_cfg); + err = ip6_convert_metrics(net, rt, r_cfg); if (err) { kfree(nh); return err; @@ -4168,7 +4107,8 @@ static int ip6_route_multipath_add(struct fib6_config *cfg, rt->fib6_nh.nh_weight = rtnh->rtnh_hops + 1; - err = ip6_route_info_append(&rt6_nh_list, rt, &r_cfg); + err = ip6_route_info_append(info->nl_net, &rt6_nh_list, + rt, &r_cfg); if (err) { dst_release_immediate(&rt->dst); goto cleanup; @@ -4186,7 +4126,7 @@ static int ip6_route_multipath_add(struct fib6_config *cfg, err_nh = NULL; list_for_each_entry(nh, &rt6_nh_list, next) { rt_last = nh->rt6_info; - err = __ip6_ins_rt(nh->rt6_info, info, &nh->mxc, extack); + err = __ip6_ins_rt(nh->rt6_info, info, extack); /* save reference to first route for notification */ if (!rt_notif && !err) rt_notif = nh->rt6_info; @@ -4235,7 +4175,6 @@ static int ip6_route_multipath_add(struct fib6_config *cfg, list_for_each_entry_safe(nh, nh_safe, &rt6_nh_list, next) { if (nh->rt6_info) dst_release_immediate(&nh->rt6_info->dst); - kfree(nh->mxc.mx); list_del(&nh->next); kfree(nh); } @@ -4409,16 +4348,14 @@ static int rt6_add_nexthop(struct sk_buff *skb, struct rt6_info *rt) return -EMSGSIZE; } -static int rt6_fill_node(struct net *net, - struct sk_buff *skb, struct rt6_info *rt, - struct in6_addr *dst, struct in6_addr *src, +static int rt6_fill_node(struct net *net, struct sk_buff *skb, + struct rt6_info *rt, struct dst_entry *dst, + struct in6_addr *dest, struct in6_addr *src, int iif, int type, u32 portid, u32 seq, unsigned int flags) { - u32 metrics[RTAX_MAX]; struct rtmsg *rtm; struct nlmsghdr *nlh; - long expires; u32 table; nlh = nlmsg_put(skb, portid, seq, type, sizeof(*rtm), flags); @@ -4446,8 +4383,8 @@ static int rt6_fill_node(struct net *net, if (rt->rt6i_flags & RTF_CACHE) rtm->rtm_flags |= RTM_F_CLONED; - if (dst) { - if (nla_put_in6_addr(skb, RTA_DST, dst)) + if (dest) { + if (nla_put_in6_addr(skb, RTA_DST, dest)) goto nla_put_failure; rtm->rtm_dst_len = 128; } else if (rtm->rtm_dst_len) @@ -4475,9 +4412,9 @@ static int rt6_fill_node(struct net *net, #endif if (nla_put_u32(skb, RTA_IIF, iif)) goto nla_put_failure; - } else if (dst) { + } else if (dest) { struct in6_addr saddr_buf; - if (ip6_route_get_saddr(net, rt, dst, 0, &saddr_buf) == 0 && + if (ip6_route_get_saddr(net, rt, dest, 0, &saddr_buf) == 0 && nla_put_in6_addr(skb, RTA_PREFSRC, &saddr_buf)) goto nla_put_failure; } @@ -4489,10 +4426,7 @@ static int rt6_fill_node(struct net *net, goto nla_put_failure; } - memcpy(metrics, dst_metrics_ptr(&rt->dst), sizeof(metrics)); - if (rt->rt6i_pmtu) - metrics[RTAX_MTU - 1] = rt->rt6i_pmtu; - if (rtnetlink_put_metrics(skb, metrics) < 0) + if (rtnetlink_put_metrics(skb, rt->fib6_metrics->metrics) < 0) goto nla_put_failure; if (nla_put_u32(skb, RTA_PRIORITY, rt->rt6i_metric)) @@ -4524,10 +4458,15 @@ static int rt6_fill_node(struct net *net, goto nla_put_failure; } - expires = (rt->rt6i_flags & RTF_EXPIRES) ? rt->dst.expires - jiffies : 0; + if (dst) { + long expires = 0; - if (rtnl_put_cacheinfo(skb, &rt->dst, 0, expires, rt->dst.error) < 0) - goto nla_put_failure; + if (rt->rt6i_flags & RTF_EXPIRES) + expires = dst->expires - jiffies; + + if (rtnl_put_cacheinfo(skb, dst, 0, expires, dst->error) < 0) + goto nla_put_failure; + } if (nla_put_u8(skb, RTA_PREF, IPV6_EXTRACT_PREF(rt->rt6i_flags))) goto nla_put_failure; @@ -4560,10 +4499,9 @@ int rt6_dump_route(struct rt6_info *rt, void *p_arg) } } - return rt6_fill_node(net, - arg->skb, rt, NULL, NULL, 0, RTM_NEWROUTE, - NETLINK_CB(arg->cb->skb).portid, arg->cb->nlh->nlmsg_seq, - NLM_F_MULTI); + return rt6_fill_node(net, arg->skb, rt, NULL, NULL, NULL, 0, + RTM_NEWROUTE, NETLINK_CB(arg->cb->skb).portid, + arg->cb->nlh->nlmsg_seq, NLM_F_MULTI); } static int inet6_rtm_getroute(struct sk_buff *in_skb, struct nlmsghdr *nlh, @@ -4677,13 +4615,14 @@ static int inet6_rtm_getroute(struct sk_buff *in_skb, struct nlmsghdr *nlh, skb_dst_set(skb, &rt->dst); if (fibmatch) - err = rt6_fill_node(net, skb, rt, NULL, NULL, iif, + err = rt6_fill_node(net, skb, rt, NULL, NULL, NULL, iif, RTM_NEWROUTE, NETLINK_CB(in_skb).portid, nlh->nlmsg_seq, 0); else - err = rt6_fill_node(net, skb, rt, &fl6.daddr, &fl6.saddr, iif, - RTM_NEWROUTE, NETLINK_CB(in_skb).portid, - nlh->nlmsg_seq, 0); + err = rt6_fill_node(net, skb, rt, dst, &fl6.daddr, &fl6.saddr, + iif, RTM_NEWROUTE, + NETLINK_CB(in_skb).portid, nlh->nlmsg_seq, + 0); if (err < 0) { kfree_skb(skb); goto errout; @@ -4709,8 +4648,8 @@ void inet6_rt_notify(int event, struct rt6_info *rt, struct nl_info *info, if (!skb) goto errout; - err = rt6_fill_node(net, skb, rt, NULL, NULL, 0, - event, info->portid, seq, nlm_flags); + err = rt6_fill_node(net, skb, rt, NULL, NULL, NULL, 0, + event, info->portid, seq, nlm_flags); if (err < 0) { /* -EMSGSIZE implies BUG in rt6_nlmsg_size() */ WARN_ON(err == -EMSGSIZE);