| Message ID | 20130731093653.GN5610@lug-owl.de |
|---|---|
| State | New |
| Headers | show |
On Wed, 2013-07-31 at 11:36 +0200, Jan-Benedict Glaw wrote: > On Wed, 2013-07-31 10:34:10 +0200, Jan-Benedict Glaw <jbglaw@lug-owl.de> wrote: > > On Tue, 2013-07-30 20:50:27 -0400, David Malcolm <dmalcolm@redhat.com> wrote: > > [...] > > > Committed to trunk as r201359, having double-checked that it > > > bootstrapped by itself on top of what had gone before, and that the > > > testsuite results were unaffected by it (on x86_64-unknown-linux-gnu). > > > > [...] > > mawk -f ../../../../gcc/gcc/gen-pass-instances.awk \ > > ../../../../gcc/gcc/passes.def > pass-instances.def > [...] > > > It seems this does only happen on one of the three running build > > clients. That one is using `mawk' instead of `gawk', what the two > > other builders (which are not affected) use. > > The substr() was wrong, awk starts all its indices with 1, while the > script used 0. gawk ignores this, mawk starts the substring at 1, but > counts from 0 upwards. > > Fixed as trivial: > > 2013-07-31 Jan-Benedict Glaw <jbglaw@owl.de> > > * gen-pass-instances.awk: Fix offset of substr(). A thousand apologies, and thanks for fixing this - I guess I owe you a $FAVORITE_BEVERAGE. I had only tested with gawk (with and without -c) and with busybox awk. I've now installed mawk and nawk on my dev box. Are that any other awks I should be testing with - and is this information captured somewhere for reference? I've verified that gawk, gawk -c, mawk, nawk and busybox awk all generate the same output on your version (r201364) thusly in bash: $ for AWK in gawk "gawk -c" mawk nawk "busybox awk" ; \ do \ $AWK -f gen-pass-instances.awk.new passes.def \ > "pass-instances-new-$AWK.def" ; \ done $ md5sum pass-instances-new-* e7d0f2c2ab4f2e83fbd032f222d66530 pass-instances-new-busybox awk.def e7d0f2c2ab4f2e83fbd032f222d66530 pass-instances-new-gawk -c.def e7d0f2c2ab4f2e83fbd032f222d66530 pass-instances-new-gawk.def e7d0f2c2ab4f2e83fbd032f222d66530 pass-instances-new-mawk.def e7d0f2c2ab4f2e83fbd032f222d66530 pass-instances-new-nawk.def However, having said that, with my original version (as of r201359), I get the same results as with your version: $ md5sum gen-pass-instances.awk.* 6dcc4e2de8241f1811435013da44b757 gen-pass-instances.awk.new 7e5ad85f1f919dea9862b420ddbb0fdd gen-pass-instances.awk.old $ diff gen-pass-instances.awk.old gen-pass-instances.awk.new 58c58 < substr(line, 0, pass_starts_at + len_of_pass_name - 1), --- > substr(line, 1, pass_starts_at + len_of_pass_name - 1), $ for AWK in gawk "gawk -c" mawk nawk "busybox awk" ; \ do \ $AWK -f gen-pass-instances.awk.old passes.def \ > "pass-instances-old-$AWK.def" ; \ done $ md5sum pass-instances-old-* e7d0f2c2ab4f2e83fbd032f222d66530 pass-instances-old-busybox awk.def e7d0f2c2ab4f2e83fbd032f222d66530 pass-instances-old-gawk -c.def e7d0f2c2ab4f2e83fbd032f222d66530 pass-instances-old-gawk.def e7d0f2c2ab4f2e83fbd032f222d66530 pass-instances-old-mawk.def e7d0f2c2ab4f2e83fbd032f222d66530 pass-instances-old-nawk.def so it seems they're all generating the same output, with both my old version and your new version. Have I messed up my testing above, or is something else going on? What version of mawk are you using? I'm using versions packaged on a Fedora 17 box: $ rpm -q gawk mawk nawk busybox gawk-4.0.1-1.fc17.x86_64 mawk-1.3.4-1.20130219.fc17.x86_64 nawk-20110810-3.fc17.x86_64 busybox-1.19.4-4.fc17.x86_64 i.e.: * GNU Awk 4.0.1 * mawk 1.3.4 * nawk --version gives "awk version 20110810" * BusyBox v1.19.4 (2012-04-18 15:11:20 UTC) multi-call binary. Thanks again Dave
On Wed, 2013-07-31 11:31:35 -0400, David Malcolm <dmalcolm@redhat.com> wrote: > On Wed, 2013-07-31 at 11:36 +0200, Jan-Benedict Glaw wrote: > > On Wed, 2013-07-31 10:34:10 +0200, Jan-Benedict Glaw <jbglaw@lug-owl.de> wrote: [breakage with mawk] > > > It seems this does only happen on one of the three running build > > > clients. That one is using `mawk' instead of `gawk', what the two > > > other builders (which are not affected) use. > > > > The substr() was wrong, awk starts all its indices with 1, while the > > script used 0. gawk ignores this, mawk starts the substring at 1, but > > counts from 0 upwards. > > A thousand apologies, and thanks for fixing this - I guess I owe you a > $FAVORITE_BEVERAGE. I had only tested with gawk (with and without -c) > and with busybox awk. I've now installed mawk and nawk on my dev box. That's why I'm running the build robot :) Once I've got some time[tm], I'll spend it a small web frontend to look into the basic information (which builds worked/failed, show build logfile, show git log between working/non-working version.) > Are that any other awks I should be testing with - and is this > information captured somewhere for reference? I don't know of any--I'm actually not a regular awk user at all. And actually testing specific awk features isn't codified IMHO. And while indices start at 1 (so the code was buggy in that way), it might also be true that the implementation-defined behavior of virtually all awk variants is to silently s/0/1/. Then, this would be an additional glitch (not bug) in my mawk. > However, having said that, with my original version (as of r201359), I > get the same results as with your version: > > $ md5sum gen-pass-instances.awk.* > 6dcc4e2de8241f1811435013da44b757 gen-pass-instances.awk.new > 7e5ad85f1f919dea9862b420ddbb0fdd gen-pass-instances.awk.old > > $ diff gen-pass-instances.awk.old gen-pass-instances.awk.new > 58c58 > < substr(line, 0, pass_starts_at + len_of_pass_name - 1), > --- > > substr(line, 1, pass_starts_at + len_of_pass_name - 1), > > $ for AWK in gawk "gawk -c" mawk nawk "busybox awk" ; \ > do \ > $AWK -f gen-pass-instances.awk.old passes.def \ > > "pass-instances-old-$AWK.def" ; \ > done > > $ md5sum pass-instances-old-* > e7d0f2c2ab4f2e83fbd032f222d66530 pass-instances-old-busybox awk.def > e7d0f2c2ab4f2e83fbd032f222d66530 pass-instances-old-gawk -c.def > e7d0f2c2ab4f2e83fbd032f222d66530 pass-instances-old-gawk.def > e7d0f2c2ab4f2e83fbd032f222d66530 pass-instances-old-mawk.def > e7d0f2c2ab4f2e83fbd032f222d66530 pass-instances-old-nawk.def > > so it seems they're all generating the same output, with both my old > version and your new version. > > Have I messed up my testing above, or is something else going on? What > version of mawk are you using? Your testing looks fine to me. My Debian "unstable" mawk is slightly older than yours: 1.3.3-17 MfG, JBG
On Wed, 2013-07-31 at 18:00 +0200, Jan-Benedict Glaw wrote: > On Wed, 2013-07-31 11:31:35 -0400, David Malcolm <dmalcolm@redhat.com> wrote: > > On Wed, 2013-07-31 at 11:36 +0200, Jan-Benedict Glaw wrote: > > > On Wed, 2013-07-31 10:34:10 +0200, Jan-Benedict Glaw <jbglaw@lug-owl.de> wrote: > [breakage with mawk] > > > > It seems this does only happen on one of the three running build > > > > clients. That one is using `mawk' instead of `gawk', what the two > > > > other builders (which are not affected) use. > > > > > > The substr() was wrong, awk starts all its indices with 1, while the > > > script used 0. gawk ignores this, mawk starts the substring at 1, but > > > counts from 0 upwards. > > > > A thousand apologies, and thanks for fixing this - I guess I owe you a > > $FAVORITE_BEVERAGE. I had only tested with gawk (with and without -c) > > and with busybox awk. I've now installed mawk and nawk on my dev box. > > That's why I'm running the build robot :) Once I've got some > time[tm], I'll spend it a small web frontend to look into the basic > information (which builds worked/failed, show build logfile, show git > log between working/non-working version.) BTW, have you seen buildbot? i.e. http://buildbot.net/ MIT-licensed and Python-based. It's in Fedora, and I see that it's in Debian: http://packages.debian.org/unstable/buildbot (With my Python hat on, we use that for verifying every commit of CPython builds and passes the testsuite, across multiple build environments, and it's fairly easy for 3rd-parties to hook in their own build slaves into a build farm, for the less common envs). > > Are that any other awks I should be testing with - and is this > > information captured somewhere for reference? > > I don't know of any--I'm actually not a regular awk user at all. > And actually testing specific awk features isn't codified IMHO. And > while indices start at 1 (so the code was buggy in that way), it might > also be true that the implementation-defined behavior of virtually all > awk variants is to silently s/0/1/. Then, this would be an additional > glitch (not bug) in my mawk. It sounds like your awk expertise is greater than mine, though :) [...snip my testing...] > > Have I messed up my testing above, or is something else going on? What > > version of mawk are you using? > > Your testing looks fine to me. My Debian "unstable" mawk is slightly > older than yours: 1.3.3-17 Looking at mawk's CHANGES file I see this entry: 20090726 [...snip...] + modify workaround for (incorrect) scripts which use a zero-parameter for substr to ensure the overall length of the result stays the same. For example, from makewhatis: filename_no_gz = substr(filename, 0, RSTART - 1); so perhaps that's it. Thanks again. Dave
On Wed, 2013-07-31 12:18:42 -0400, David Malcolm <dmalcolm@redhat.com> wrote: > On Wed, 2013-07-31 at 18:00 +0200, Jan-Benedict Glaw wrote: > > That's why I'm running the build robot :) Once I've got some > > time[tm], I'll spend it a small web frontend to look into the basic > > information (which builds worked/failed, show build logfile, show git > > log between working/non-working version.) > > BTW, have you seen buildbot? i.e. http://buildbot.net/ > > MIT-licensed and Python-based. Yes, I have! Though I'm not yet enough of a Python wizard to build me a working configuration the way I wanted to have it. It's like 400 LoC shell script right now (the build master), plus a shell script doing the actual building on the clients (160 LoC). It basically builds as many crosscompilers on all nodes as possible, until a new commit shows up. And once a new commit shows up, new build jobs are only triggered if they're expected to finish before the last of all currently jobs is expected to be done. Then, all builders are updated and start building again. > It's in Fedora, and I see that it's in Debian: > http://packages.debian.org/unstable/buildbot > > (With my Python hat on, we use that for verifying every commit of > CPython builds and passes the testsuite, across multiple build > environments, and it's fairly easy for 3rd-parties to hook in their own > build slaves into a build farm, for the less common envs). If you have a few hours, you'd help substituting my scripts with something that's possibly better cared for upstream :) > > I don't know of any--I'm actually not a regular awk user at all. > > And actually testing specific awk features isn't codified IMHO. And > > while indices start at 1 (so the code was buggy in that way), it might > > also be true that the implementation-defined behavior of virtually all > > awk variants is to silently s/0/1/. Then, this would be an additional > > glitch (not bug) in my mawk. > > It sounds like your awk expertise is greater than mine, though :) I just read the docs after verifying that the helper variables were all common between gawk and mawk. > 20090726 > [...snip...] > + modify workaround for (incorrect) scripts which use a zero-parameter > for substr to ensure the overall length of the result stays the same. > For example, from makewhatis: > filename_no_gz = substr(filename, 0, RSTART - 1); > > so perhaps that's it. Probably it is. MfG, JBG
Index: gcc/gen-pass-instances.awk =================================================================== --- gcc/gen-pass-instances.awk (revision 201363) +++ gcc/gen-pass-instances.awk (working copy) @@ -55,7 +55,7 @@ else pass_counts[pass_name] = 1; printf "%s, %s%s\n", - substr(line, 0, pass_starts_at + len_of_pass_name - 1), + substr(line, 1, pass_starts_at + len_of_pass_name - 1), pass_counts[pass_name], substr(line, pass_starts_at + len_of_pass_name); } else {