posix: Make tst-getaddrinfo5 an xtest due to Internet requirement [BZ #20826]

Message ID 20171113132213.4127641274EC9@oldenburg.str.redhat.com
State New
Headers show
Series
  • posix: Make tst-getaddrinfo5 an xtest due to Internet requirement [BZ #20826]
Related show

Commit Message

Florian Weimer Nov. 13, 2017, 1:22 p.m.
We have additional coverage of getaddrinfo through the resolv tests, so
the loss of default test coverage seems acceptable.

2017-11-13  Florian Weimer  <fweimer@redhat.com>

	[BZ #20826]
	Turn posix/tst-getaddrinfo5 into xtest due to Internet requirement.
	* posix/Makefile (tests): Remove tst-getaddrinfo5.
	(xtests): Add tst-getaddrinfo5.

Comments

Aurelien Jarno Nov. 14, 2017, 8:17 p.m. | #1
On 2017-11-13 14:22, Florian Weimer wrote:
> We have additional coverage of getaddrinfo through the resolv tests, so
> the loss of default test coverage seems acceptable.
> 
> 2017-11-13  Florian Weimer  <fweimer@redhat.com>
> 
> 	[BZ #20826]
> 	Turn posix/tst-getaddrinfo5 into xtest due to Internet requirement.
> 	* posix/Makefile (tests): Remove tst-getaddrinfo5.
> 	(xtests): Add tst-getaddrinfo5.
> 

IMHO, this test should succeed even without network access, provided
that /etc/hosts contains an entry for localhost, which is something
pretty standard. However nss_files only consider exact matching, and
doesn't consider 'localhost.' being the same as 'localhost'.
Carlos O'Donell Nov. 14, 2017, 11:43 p.m. | #2
On 11/14/2017 12:17 PM, Aurelien Jarno wrote:
> On 2017-11-13 14:22, Florian Weimer wrote:
>> We have additional coverage of getaddrinfo through the resolv tests, so
>> the loss of default test coverage seems acceptable.
>>
>> 2017-11-13  Florian Weimer  <fweimer@redhat.com>
>>
>> 	[BZ #20826]
>> 	Turn posix/tst-getaddrinfo5 into xtest due to Internet requirement.
>> 	* posix/Makefile (tests): Remove tst-getaddrinfo5.
>> 	(xtests): Add tst-getaddrinfo5.
>>
> 
> IMHO, this test should succeed even without network access, provided
> that /etc/hosts contains an entry for localhost, which is something
> pretty standard. However nss_files only consider exact matching, and
> doesn't consider 'localhost.' being the same as 'localhost'.
 
We will eventually run this test in a newtork namespace with a known
configured hosts file. Until that point I think it needs to remain an
xtest. We should not rely on the host being configured in any magical
way.

DJ is currently looking at running tests in a minimal namespace to
allow us to do exactly this kind of testing. If you have any interest
in that please reach out to DJ.
Andreas Schwab Nov. 15, 2017, 7:54 a.m. | #3
On Nov 13 2017, fweimer@redhat.com (Florian Weimer) wrote:

> We have additional coverage of getaddrinfo through the resolv tests, so
> the loss of default test coverage seems acceptable.
>
> 2017-11-13  Florian Weimer  <fweimer@redhat.com>
>
> 	[BZ #20826]
> 	Turn posix/tst-getaddrinfo5 into xtest due to Internet requirement.
> 	* posix/Makefile (tests): Remove tst-getaddrinfo5.
> 	(xtests): Add tst-getaddrinfo5.

tst-getaddrinfo4 has the same problem.

Andreas.
Florian Weimer Nov. 15, 2017, 7:54 a.m. | #4
On 11/15/2017 08:54 AM, Andreas Schwab wrote:
> On Nov 13 2017, fweimer@redhat.com (Florian Weimer) wrote:
> 
>> We have additional coverage of getaddrinfo through the resolv tests, so
>> the loss of default test coverage seems acceptable.
>>
>> 2017-11-13  Florian Weimer  <fweimer@redhat.com>
>>
>> 	[BZ #20826]
>> 	Turn posix/tst-getaddrinfo5 into xtest due to Internet requirement.
>> 	* posix/Makefile (tests): Remove tst-getaddrinfo5.
>> 	(xtests): Add tst-getaddrinfo5.
> 
> tst-getaddrinfo4 has the same problem.

Should we turn both into xtests?

Thanks,
Florian
Andreas Schwab Nov. 15, 2017, 8:30 a.m. | #5
On Nov 15 2017, Florian Weimer <fweimer@redhat.com> wrote:

> On 11/15/2017 08:54 AM, Andreas Schwab wrote:
>> On Nov 13 2017, fweimer@redhat.com (Florian Weimer) wrote:
>>
>>> We have additional coverage of getaddrinfo through the resolv tests, so
>>> the loss of default test coverage seems acceptable.
>>>
>>> 2017-11-13  Florian Weimer  <fweimer@redhat.com>
>>>
>>> 	[BZ #20826]
>>> 	Turn posix/tst-getaddrinfo5 into xtest due to Internet requirement.
>>> 	* posix/Makefile (tests): Remove tst-getaddrinfo5.
>>> 	(xtests): Add tst-getaddrinfo5.
>>
>> tst-getaddrinfo4 has the same problem.
>
> Should we turn both into xtests?

Either both or neither.

Andreas.
Florian Weimer Nov. 15, 2017, 10:29 a.m. | #6
On 11/15/2017 09:30 AM, Andreas Schwab wrote:
> On Nov 15 2017, Florian Weimer <fweimer@redhat.com> wrote:
> 
>> On 11/15/2017 08:54 AM, Andreas Schwab wrote:
>>> On Nov 13 2017, fweimer@redhat.com (Florian Weimer) wrote:
>>>
>>>> We have additional coverage of getaddrinfo through the resolv tests, so
>>>> the loss of default test coverage seems acceptable.
>>>>
>>>> 2017-11-13  Florian Weimer  <fweimer@redhat.com>
>>>>
>>>> 	[BZ #20826]
>>>> 	Turn posix/tst-getaddrinfo5 into xtest due to Internet requirement.
>>>> 	* posix/Makefile (tests): Remove tst-getaddrinfo5.
>>>> 	(xtests): Add tst-getaddrinfo5.
>>>
>>> tst-getaddrinfo4 has the same problem.
>>
>> Should we turn both into xtests?
> 
> Either both or neither.

Fair point.  I'm attaching an updated patch.

Thanks,
Florian
These tests need a working Internet connection with DNS.

We have additional coverage of getaddrinfo through the resolv tests, so
the loss of default test coverage seems acceptable.

2017-11-15  Florian Weimer  <fweimer@redhat.com>

	[BZ #20826]
	Turn posix/tst-getaddrinfo4, posix/tst-getaddrinfo5 into xtests
	due to Internet requirement.
	* posix/Makefile (tests): Remove tst-getaddrinfo4,
	tst-getaddrinfo5.
	(xtests): Add tst-getaddrinfo4, tst-getaddrinfo5.

diff --git a/posix/Makefile b/posix/Makefile
index 532f2c73c5..87e7ed7f9d 100644
--- a/posix/Makefile
+++ b/posix/Makefile
@@ -90,8 +90,8 @@ tests		:= test-errno tstgetopt testfnm runtests runptests \
 		   tst-getaddrinfo3 tst-fnmatch2 tst-cpucount tst-cpuset \
 		   bug-getopt1 bug-getopt2 bug-getopt3 bug-getopt4 \
 		   bug-getopt5 tst-getopt_long1 bug-regex34 bug-regex35 \
-		   tst-pathconf tst-getaddrinfo4 tst-rxspencer-no-utf8 \
-		   tst-fnmatch3 bug-regex36 tst-getaddrinfo5 \
+		   tst-pathconf tst-rxspencer-no-utf8 \
+		   tst-fnmatch3 bug-regex36 \
 		   tst-posix_spawn-fd tst-posix_spawn-setsid \
 		   tst-posix_fadvise tst-posix_fadvise64 \
 		   tst-sysconf-empty-chroot tst-glob_symlinks tst-fexecve \
@@ -99,7 +99,7 @@ tests		:= test-errno tstgetopt testfnm runtests runptests \
 tests-internal	:= bug-regex5 bug-regex20 bug-regex33 \
 		   tst-rfc3484 tst-rfc3484-2 tst-rfc3484-3 \
 		   tst-glob_lstat_compat
-xtests		:= bug-ga2
+xtests		:= bug-ga2 tst-getaddrinfo4 tst-getaddrinfo5
 ifeq (yes,$(build-shared))
 test-srcs	:= globtest
 tests           += wordexp-test tst-exec tst-spawn tst-spawn2 tst-spawn3

Patch

diff --git a/posix/Makefile b/posix/Makefile
index 532f2c73c5..8b3632a3ba 100644
--- a/posix/Makefile
+++ b/posix/Makefile
@@ -91,7 +91,7 @@  tests		:= test-errno tstgetopt testfnm runtests runptests \
 		   bug-getopt1 bug-getopt2 bug-getopt3 bug-getopt4 \
 		   bug-getopt5 tst-getopt_long1 bug-regex34 bug-regex35 \
 		   tst-pathconf tst-getaddrinfo4 tst-rxspencer-no-utf8 \
-		   tst-fnmatch3 bug-regex36 tst-getaddrinfo5 \
+		   tst-fnmatch3 bug-regex36 \
 		   tst-posix_spawn-fd tst-posix_spawn-setsid \
 		   tst-posix_fadvise tst-posix_fadvise64 \
 		   tst-sysconf-empty-chroot tst-glob_symlinks tst-fexecve \
@@ -99,7 +99,7 @@  tests		:= test-errno tstgetopt testfnm runtests runptests \
 tests-internal	:= bug-regex5 bug-regex20 bug-regex33 \
 		   tst-rfc3484 tst-rfc3484-2 tst-rfc3484-3 \
 		   tst-glob_lstat_compat
-xtests		:= bug-ga2
+xtests		:= bug-ga2 tst-getaddrinfo5
 ifeq (yes,$(build-shared))
 test-srcs	:= globtest
 tests           += wordexp-test tst-exec tst-spawn tst-spawn2 tst-spawn3