Message ID | 20160615154119.GS4053@vapier.lan |
---|---|
State | New |
Headers | show |
On 06/15/2016 05:41 PM, Mike Frysinger wrote: >> I think we should really have some sort of libtest.a, which provides >> test helpers, error-checking functions such as xmalloc, and general >> helpers for setting up special test environments. I'm a bit worried >> about figuring out the proper dependencies, so that libtest.a is built >> before all the tests are linked. > > isn't this what test-skeleton.c does for us now ? and we all agree that > all tests should be using that. There are limits to what we can put into test-skeleton.c due to symbol conflicts and dependencies of the test harness which are incompatible with some things which we want to test. With libtest.a, we can still have clean compilation environments for tests, and the linker will sort out things for us. (We can even have ELF constructors which get pulled into the link as needed.) > imo we should just mandate all tests use the entry point "do_test" and it > take argc/argv args (even if they're unused), and then all tests can pull > test-skeleton.c in at the top. Yes, seems a reasonable improvement, and mostly unrelated. Thanks, Florian
On 22 Jun 2016 11:57, Florian Weimer wrote: > On 06/15/2016 05:41 PM, Mike Frysinger wrote: > >> I think we should really have some sort of libtest.a, which provides > >> test helpers, error-checking functions such as xmalloc, and general > >> helpers for setting up special test environments. I'm a bit worried > >> about figuring out the proper dependencies, so that libtest.a is built > >> before all the tests are linked. > > > > isn't this what test-skeleton.c does for us now ? and we all agree that > > all tests should be using that. > > There are limits to what we can put into test-skeleton.c due to symbol > conflicts and dependencies of the test harness which are incompatible > with some things which we want to test. symbol-wise, i don't think there's a realistic problem. we've rarely (ever?) run into this problem, and it's trivial to sort out by using a namespace prefix like "tst_". which we probably want to do anyways to keep the API more readable. dependency wise (e.g. dl or pthread usage), i'd wait until the need actually arises before i'd start worrying about it. > With libtest.a, we can still have clean compilation environments for > tests, and the linker will sort out things for us. (We can even have > ELF constructors which get pulled into the link as needed.) i don't think i'd agree with that. if our test API used tst_xxx and random tests started interposing their own variant, that's just ugly and a pita to digest to the point where it shouldn't really be allowed. > > imo we should just mandate all tests use the entry point "do_test" and it > > take argc/argv args (even if they're unused), and then all tests can pull > > test-skeleton.c in at the top. > > Yes, seems a reasonable improvement, and mostly unrelated. they are related when you look at the implications of the two paths. if we have a single test-skeleton.c, having a sep header for all the prototypes is kind of pointless and just busy work. but if there's a lib of test files, then having a sep header is pretty much required. if there isn't a sep header, then the include point of the skeleton must be at the top since it's acting as the header. -mike
On 06/22/2016 01:05 PM, Mike Frysinger wrote: > On 22 Jun 2016 11:57, Florian Weimer wrote: >> On 06/15/2016 05:41 PM, Mike Frysinger wrote: >>>> I think we should really have some sort of libtest.a, which provides >>>> test helpers, error-checking functions such as xmalloc, and general >>>> helpers for setting up special test environments. I'm a bit worried >>>> about figuring out the proper dependencies, so that libtest.a is built >>>> before all the tests are linked. >>> >>> isn't this what test-skeleton.c does for us now ? and we all agree that >>> all tests should be using that. >> >> There are limits to what we can put into test-skeleton.c due to symbol >> conflicts and dependencies of the test harness which are incompatible >> with some things which we want to test. > > symbol-wise, i don't think there's a realistic problem. we've rarely > (ever?) run into this problem, and it's trivial to sort out by using > a namespace prefix like "tst_". which we probably want to do anyways > to keep the API more readable. What I mean is that the test harness pulls in stuff that cause things to interfere with what we want to test. This could be magic symbols for stdio/libio compatibility, pthread symbols, or just calling mallopt (as discussed before). If the test helpers are not in just one monolithic .c file, it helps with achieving that. > dependency wise (e.g. dl or pthread usage), i'd wait until the need > actually arises before i'd start worrying about it. The need for pthread error-checking wrappers is already there. Our current policy is to check error returns from *all* pthread functions, and getting a nice error from the usually looks like this: int ret = ...; if (ret != 0) { errno = ret; printf ("error: ...: %m\n"); _exit (1); } with some special exceptions such us pthread_barrier_wait. Many tests are littered with that. >>> imo we should just mandate all tests use the entry point "do_test" and it >>> take argc/argv args (even if they're unused), and then all tests can pull >>> test-skeleton.c in at the top. >> >> Yes, seems a reasonable improvement, and mostly unrelated. > > they are related when you look at the implications of the two paths. > if we have a single test-skeleton.c, having a sep header for all the > prototypes is kind of pointless and just busy work. but if there's a > lib of test files, then having a sep header is pretty much required. > if there isn't a sep header, then the include point of the skeleton > must be at the top since it's acting as the header. In both cases, we want the include at the top, so the change your proposed goes in the right direction, whether we want a libtest.a or not. Thanks, Florian
On Thu, 23 Jun 2016, Florian Weimer wrote: > What I mean is that the test harness pulls in stuff that cause things to > interfere with what we want to test. This could be magic symbols for > stdio/libio compatibility, pthread symbols, or just calling mallopt (as > discussed before). > > If the test helpers are not in just one monolithic .c file, it helps with > achieving that. Also, test-skeleton.c doesn't work well with tests that wish to undefine _LIBC and _GNU_SOURCE in order to test non-GNU feature test macros / APIs / strict standards conformance options, since it uses GNU APIs itself. If we change things so that most of the functionality is in a separate library or object (built with the usual _GNU_SOURCE), and test-skeleton itself is very minimal, then more tests can be made to use it. (Of course, doing this means doing something about the various macros tests can define to change how test-skeleton behaves.)
--- a/test-skeleton.c +++ b/test-skeleton.c @@ -343,6 +343,10 @@ main (int argc, char *argv[]) setbuf (stdout, NULL); #endif + fclose (stderr); + dup2 (STDOUT_FILENO, STDERR_FILENO); + stderr = fdopen (STDERR_FILENO, "w"); + while ((opt = getopt_long (argc, argv, "+", options, NULL)) != -1) switch (opt) {