diff mbox

[testsuite] Fix profile test failures

Message ID 2e4e3d80-6505-4f09-8a24-01e99dcda418@BAMAIL02.ba.imgtec.org
State New
Headers show

Commit Message

Steve Ellcey Feb. 13, 2014, 10:08 p.m. UTC
While testing the C++ profiling tests in g++.dg/bprob and using the
qemu simulator we discovered that these tests were passing when we ran
the testsuite with no extra options but that if we specified some options
on the testsuite run then the tests would fail with this message in the
c++.log file:

rsh: Could not resolve hostname multi-sim/-EL: Name or service not known

After some poking around I found that profopt-execute in lib/profopt.exp
was using remote_file and remote_upload with 'target' where I believe it 
should be using 'host'.  No other *.exp file uses 'target' on their
remote_file or remote_update calls, they either use 'build' or 'host'.

So while it seems weird that 'host' is the proper replacement for 'target'
as the machine where the executable is run, this seems to be the right fix
and it does give me a clean run now with or without extra arguments on
the test run.

OK for checkin?

Steve Ellcey
sellcey@mips.com


2014-02-13  Steve Ellcey <sellcey@mips.com>

	* lib/profopt.exp (profopt-execute): Use host instead of target
	in remote_file and remote_upload calls.

Comments

Joseph Myers Feb. 13, 2014, 11:09 p.m. UTC | #1
On Thu, 13 Feb 2014, Steve Ellcey  wrote:

> While testing the C++ profiling tests in g++.dg/bprob and using the
> qemu simulator we discovered that these tests were passing when we ran
> the testsuite with no extra options but that if we specified some options
> on the testsuite run then the tests would fail with this message in the
> c++.log file:
> 
> rsh: Could not resolve hostname multi-sim/-EL: Name or service not known

That means your board file is buggy.  If rsh is not the right way to 
access your target system, you need to implement the board file methods in 
some way other than rsh (possibly some operations should be no-ops, or do 
something directly on the build system, if you have a shared filesystem).

> So while it seems weird that 'host' is the proper replacement for 'target'
> as the machine where the executable is run, this seems to be the right fix

It's certainly not the proper replacement.  If a file is on the target, 
use target for deletion / manipulation; if it's on the host, use host for 
deletion / manipulation; on build, use build; in multiple places, run the 
deletion operation once per system with the file; to copy from target to 
the system (build) running DejaGnu, use remote_upload specifying target; 
to copy from host to build, use remote_upload specifying host; to copy 
from build to host or target, use remote_download specifying host or 
target as appropriate.

To determine whether anything should be changed in a GCC .exp file, reason 
about which of the three systems (build, host, target) a file is on, or is 
needed on, at each point, rather than looking at what does or does not 
work with a buggy board file.
Steve Ellcey Feb. 13, 2014, 11:49 p.m. UTC | #2
On Thu, 2014-02-13 at 23:09 +0000, Joseph S. Myers wrote:
> On Thu, 13 Feb 2014, Steve Ellcey  wrote:
> 
> > While testing the C++ profiling tests in g++.dg/bprob and using the
> > qemu simulator we discovered that these tests were passing when we ran
> > the testsuite with no extra options but that if we specified some options
> > on the testsuite run then the tests would fail with this message in the
> > c++.log file:
> > 
> > rsh: Could not resolve hostname multi-sim/-EL: Name or service not known
> 
> That means your board file is buggy.  If rsh is not the right way to 
> access your target system, you need to implement the board file methods in 
> some way other than rsh (possibly some operations should be no-ops, or do 
> something directly on the build system, if you have a shared filesystem).

I thought the bug was that it was using 'multi-sim/-EL' instead of just
'multi-sim'.  I.e.  I thought that target was a combination of where the
test was run and what options were used, whereas host was just going to
be where the test was run.  I guess I was wrong about that.

> > So while it seems weird that 'host' is the proper replacement for 'target'
> > as the machine where the executable is run, this seems to be the right fix
> 
> It's certainly not the proper replacement.  If a file is on the target, 
> use target for deletion / manipulation; if it's on the host, use host for 
> deletion / manipulation; on build, use build; in multiple places, run the 
> deletion operation once per system with the file; to copy from target to 
> the system (build) running DejaGnu, use remote_upload specifying target; 
> to copy from host to build, use remote_upload specifying host; to copy 
> from build to host or target, use remote_download specifying host or 
> target as appropriate.

So let me make sure I understand this:  host is where you run the
testsuite from, build is where the compilation happens (probably the
same as host for most people), and target is where the test program is
executed.

> To determine whether anything should be changed in a GCC .exp file, reason 
> about which of the three systems (build, host, target) a file is on, or is 
> needed on, at each point, rather than looking at what does or does not 
> work with a buggy board file.

I am not convinced that the problem is in the board file because the
only tests I see fail this way are the ones in g++.exp/bprob and that is
also the only GCC .exp file that uses remote_upload or remote_file
with 'target'.  I will dig into it some more and also try it with some
different boards.

Steve Ellcey
sellcey@mips.com
Joseph Myers Feb. 14, 2014, 12:25 a.m. UTC | #3
On Thu, 13 Feb 2014, Steve Ellcey wrote:

> So let me make sure I understand this:  host is where you run the
> testsuite from, build is where the compilation happens (probably the
> same as host for most people), and target is where the test program is
> executed.

Host is the system on which the compilers being tested run.  Build is the 
system on which runtest runs and executes the .exp files.  They are only 
different in the case of remote-host testing (using DejaGnu on GNU/Linux 
to test a compiler for Windows host, for example) - typically the same 
cases in which a Canadian cross compiler is built.

> > To determine whether anything should be changed in a GCC .exp file, reason 
> > about which of the three systems (build, host, target) a file is on, or is 
> > needed on, at each point, rather than looking at what does or does not 
> > work with a buggy board file.
> 
> I am not convinced that the problem is in the board file because the
> only tests I see fail this way are the ones in g++.exp/bprob and that is
> also the only GCC .exp file that uses remote_upload or remote_file
> with 'target'.  I will dig into it some more and also try it with some
> different boards.

Branch profiling involves the generated executables creating files with 
profile information when they run, so those files (on the target) need 
manipulating.  Most testsuites do not involve testcases generating any 
files.  But the libstdc++ testsuite uses remote_download to transfer files 
to the target, because various testcases need to open and read input 
files.
Richard Sandiford Feb. 14, 2014, 11:12 a.m. UTC | #4
Steve Ellcey <sellcey@mips.com> writes:
> On Thu, 2014-02-13 at 23:09 +0000, Joseph S. Myers wrote:
>> On Thu, 13 Feb 2014, Steve Ellcey  wrote:
>> 
>> > While testing the C++ profiling tests in g++.dg/bprob and using the
>> > qemu simulator we discovered that these tests were passing when we ran
>> > the testsuite with no extra options but that if we specified some options
>> > on the testsuite run then the tests would fail with this message in the
>> > c++.log file:
>> > 
>> > rsh: Could not resolve hostname multi-sim/-EL: Name or service not known
>> 
>> That means your board file is buggy.  If rsh is not the right way to 
>> access your target system, you need to implement the board file methods in 
>> some way other than rsh (possibly some operations should be no-ops, or do 
>> something directly on the build system, if you have a shared filesystem).
>
> I thought the bug was that it was using 'multi-sim/-EL' instead of just
> 'multi-sim'.  I.e.  I thought that target was a combination of where the
> test was run and what options were used, whereas host was just going to
> be where the test was run.  I guess I was wrong about that.

Using target in itself should be OK.  The hostname for rsh/ssh should be
[board_info $board hostname] rather than $board itself.  So in this case
[board_info "multi-sim/-EL" hostname] should be multi-sim.  The usual way
to set that up is to put:

set_board_info hostname multi-sim

in multi-sim.exp.

Thanks,
Richard
Steve Ellcey Feb. 14, 2014, 5:24 p.m. UTC | #5
On Fri, 2014-02-14 at 11:12 +0000, Richard Sandiford wrote:

> Using target in itself should be OK.  The hostname for rsh/ssh should be
> [board_info $board hostname] rather than $board itself.  So in this case
> [board_info "multi-sim/-EL" hostname] should be multi-sim.  The usual way
> to set that up is to put:
> 
> set_board_info hostname multi-sim
> 
> in multi-sim.exp.
> 
> Thanks,
> Richard

Richard and Joseph, thanks for the pointers.  I tried Richards change
and it partially fixed the problem.  My board started doing an rsh/ssh
to 'multi-sim' instead of 'multi-sim/-EL' but that command times out
because multi-sim is not a machine that I can rsh/ssh to, it is just the
name of my dejagnu baseboard.  Looking around some more I found sim.exp
in dejagnu (which I include in my board) and I see it has overrides for
load, upload, and download but no override for exec.  I added this to
config/sim.exp:

proc sim_exec { dest srcfile args } {
    return [remote_exec host $srcfile $args]
}

to match the sim_upload and sim_download definitions and things seem to
work now.  In fact, now that I have this change to sim.exp I no longer
need to set hostname in multi-sim.exp.

I will submit the config/sim.exp patch to the dejagnu mailing list later
today.

Steve Ellcey
sellcey@mips.com
diff mbox

Patch

diff --git a/gcc/testsuite/lib/profopt.exp b/gcc/testsuite/lib/profopt.exp
index e0d849e..e045b53 100644
--- a/gcc/testsuite/lib/profopt.exp
+++ b/gcc/testsuite/lib/profopt.exp
@@ -264,7 +264,7 @@  proc profopt-execute { src } {
 
 	# Remove old profiling and performance data files.
 	foreach ext $prof_ext {
-	    remote_file target delete $tmpdir/$base.$ext
+	    remote_file host delete $tmpdir/$base.$ext
 	}
 	if [info exists perf_ext] {
 	    profopt-cleanup $testcase $perf_ext
@@ -312,7 +312,7 @@  proc profopt-execute { src } {
 	# Make sure the profile data was generated, and fail if not.
 	if { $status == "pass" } {
 	    foreach ext $prof_ext {
-		remote_upload target $tmpdir/$base.$ext
+		remote_upload host $tmpdir/$base.$ext
 		set files [glob -nocomplain $base.$ext]
 		if { $files == "" } {
 		    set status "fail"
@@ -368,7 +368,7 @@  proc profopt-execute { src } {
 
 	# Remove the profiling data files.
 	foreach ext $prof_ext {
-	    remote_file target delete $tmpdir/$base.$ext
+	    remote_file host delete $tmpdir/$base.$ext
 	}
 
 	if { $status != "pass" } {