Patchwork [regression] 3.0-rc boot failure -- bisected to cd4ea6ae3982

login
register
mail settings
Submitter Anton Blanchard
Date July 20, 2011, 10:14 a.m.
Message ID <20110720201436.19e9689a@kryten>
Download mbox | patch
Permalink /patch/105622/
State Not Applicable
Headers show

Comments

Anton Blanchard - July 20, 2011, 10:14 a.m.
Hi Peter,

> That looks very strange indeed.. up to node 23 there is the normal
> symmetric matrix with all the trace elements on 10 (as we would expect
> for local access), and some 4x4 sub-matrix stacked around the trace
> with 20, suggesting a single hop distance, and the rest on 40 being
> out-there.

I retested with the latest version of numactl, and get correct results.

I worked out why the patches don't boot, we weren't allocating any
space for the cpumask and ran off the end of the allocation.

Should we also use cpumask_copy instead of open coding it? I added that
too.

Anton
Peter Zijlstra - July 20, 2011, 10:45 a.m.
On Wed, 2011-07-20 at 20:14 +1000, Anton Blanchard wrote:

> > That looks very strange indeed.. up to node 23 there is the normal
> > symmetric matrix with all the trace elements on 10 (as we would expect
> > for local access), and some 4x4 sub-matrix stacked around the trace
> > with 20, suggesting a single hop distance, and the rest on 40 being
> > out-there.
> 
> I retested with the latest version of numactl, and get correct results.

One less thing to worry about ;-)

> I worked out why the patches don't boot, we weren't allocating any
> space for the cpumask and ran off the end of the allocation.

Gah! that's not the first time I made that particular mistake :/

> Should we also use cpumask_copy instead of open coding it? I added that
> too.

Probably, I looked for cpumask_assign() and on failing to find that used
the direct assignment.

So with that fix the patch makes the machine happy again?

Thanks!
Anton Blanchard - July 20, 2011, 12:14 p.m.
Hi Peter,

> So with that fix the patch makes the machine happy again?

Yes, the machine looks fine with the patches applied. Thanks!

Anton
Linus Torvalds - July 20, 2011, 2:40 p.m.
On Wed, Jul 20, 2011 at 5:14 AM, Anton Blanchard <anton@samba.org> wrote:
>
>> So with that fix the patch makes the machine happy again?
>
> Yes, the machine looks fine with the patches applied. Thanks!

Ok, so what's the situation for 3.0 (I'm waiting for some RCU
resolution now)? Anton's patch may be small, but that's just the tiny
fixup patch to Peter's much scarier one ;)

                            Linus
Peter Zijlstra - July 20, 2011, 2:58 p.m.
On Wed, 2011-07-20 at 07:40 -0700, Linus Torvalds wrote:
> On Wed, Jul 20, 2011 at 5:14 AM, Anton Blanchard <anton@samba.org> wrote:
> >
> >> So with that fix the patch makes the machine happy again?
> >
> > Yes, the machine looks fine with the patches applied. Thanks!
> 
> Ok, so what's the situation for 3.0 (I'm waiting for some RCU
> resolution now)? Anton's patch may be small, but that's just the tiny
> fixup patch to Peter's much scarier one ;)

Right, so we can either merge my scary patches now and have 3.0 boot on
16+ node machines (and risk breaking something), or delay them until
3.0.1 and have 16+ node machines suffer a little.

The alternative quick hack is simply to disable the node domain, but
that'll be detrimental to regular machines in that the top domain used
to have NODE sd_flags will now have ALL_NODE sd_flags which are much
less aggressive.
Linus Torvalds - July 20, 2011, 4:04 p.m.
On Wed, Jul 20, 2011 at 7:58 AM, Peter Zijlstra <a.p.zijlstra@chello.nl> wrote:
>
> Right, so we can either merge my scary patches now and have 3.0 boot on
> 16+ node machines (and risk breaking something), or delay them until
> 3.0.1 and have 16+ node machines suffer a little.

So how much impact does your scary patch have on machines that don't
have multiple nodes? If it's a "the code isn't even called by normal
machines" kind of setup, I don't think I care a lot.

                  Linus
Ingo Molnar - July 20, 2011, 4:42 p.m.
* Linus Torvalds <torvalds@linux-foundation.org> wrote:

> On Wed, Jul 20, 2011 at 7:58 AM, Peter Zijlstra <a.p.zijlstra@chello.nl> wrote:
> >
> > Right, so we can either merge my scary patches now and have 3.0 
> > boot on 16+ node machines (and risk breaking something), or delay 
> > them until 3.0.1 and have 16+ node machines suffer a little.
> 
> So how much impact does your scary patch have on machines that 
> don't have multiple nodes? If it's a "the code isn't even called by 
> normal machines" kind of setup, I don't think I care a lot.

NUMA systems will trigger the new code - not just 'weird NUMA 
systems' - but i still think we could try the patches, the code looks 
straightforward and i booted them on NUMA systems and it all seems 
fine so far.

Anyway, i'll push the new sched/urgent branch out in a few minutes 
and then you'll see the full patches in the commit notifications.

Thanks,

	Ingo

Patch

Index: linux-2.6/kernel/sched.c
===================================================================
--- linux-2.6.orig/kernel/sched.c	2011-07-20 01:54:08.191668781 -0500
+++ linux-2.6/kernel/sched.c	2011-07-20 04:45:36.203750525 -0500
@@ -7020,8 +7020,8 @@ 
 		if (cpumask_test_cpu(i, covered))
 			continue;
 
-		sg = kzalloc_node(sizeof(struct sched_group), GFP_KERNEL,
-				cpu_to_node(i));
+		sg = kzalloc_node(sizeof(struct sched_group) + cpumask_size(),
+				  GFP_KERNEL, cpu_to_node(i));
 
 		if (!sg)
 			goto fail;
@@ -7031,7 +7031,7 @@ 
 		child = *per_cpu_ptr(sdd->sd, i);
 		if (child->child) {
 			child = child->child;
-			*sg_span = *sched_domain_span(child);
+			cpumask_copy(sg_span, sched_domain_span(child));
 		} else
 			cpumask_set_cpu(i, sg_span);