Create policy for testing unstable kernel features

It's not clear if anything more than this is required for now.

Signed-off-by: Richard Palethorpe <rpalethorpe@suse.com>
---

 doc/test-writing-guidelines.txt | 10 ++++++++++
 runtest/staging                 |  1 +
 2 files changed, 11 insertions(+)
 create mode 100644 runtest/staging

Hi Richie,

Thanks!
I'd consider name next, but does not really matter that much.

+1 for single file, because not only syscalls have tests for RC
(at least IMA tests also do).

Acked-by: Petr Vorel <pvorel@suse.cz>

Kind regards,
Petr

On Thu, Feb 3, 2022 at 11:18 AM Richard Palethorpe via ltp
<ltp@lists.linux.it> wrote:
>
> It's not clear if anything more than this is required for now.
>
> Signed-off-by: Richard Palethorpe <rpalethorpe@suse.com>

Acked-by: Jan Stancek <jstancek@redhat.com>

> ---
>
>  doc/test-writing-guidelines.txt | 10 ++++++++++
>  runtest/staging                 |  1 +
>  2 files changed, 11 insertions(+)
>  create mode 100644 runtest/staging
>
> diff --git a/doc/test-writing-guidelines.txt b/doc/test-writing-guidelines.txt
> index 1fa751ed5..8ca014d45 100644
> --- a/doc/test-writing-guidelines.txt
> +++ b/doc/test-writing-guidelines.txt
> @@ -334,3 +334,13 @@ It's easier to maintain a '.gitignore' file per directory with tests, rather
>  than having single file in the project root directory. This way, we don't have
>  to update all the gitignore files when moving directories, and they get deleted
>  automatically when a directory with tests is removed.
> +
> +7 Testing pre-release kernel features
> +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
> +
> +Tests for features not yet in a mainline kernel release are accepted. However
> +they must only be added to the +staging+ runtest file. Once a feature is part
> +of the stable kernel ABI the associated test must be moved out of staging.
> +
> +This is primarily to help test kernel RCs by avoiding the need to download
> +separate LTP patchsets.
> diff --git a/runtest/staging b/runtest/staging
> new file mode 100644
> index 000000000..ef1cdea15
> --- /dev/null
> +++ b/runtest/staging
> @@ -0,0 +1 @@
> +# Tests for features that are not yet in the stable kernel ABI
> --
> 2.34.1
>
>
> --
> Mailing list info: https://lists.linux.it/listinfo/ltp
>

On Thu, Feb 3, 2022 at 6:20 PM Richard Palethorpe <rpalethorpe@suse.com>
wrote:

> It's not clear if anything more than this is required for now.
>
> Signed-off-by: Richard Palethorpe <rpalethorpe@suse.com>
>

Reviewed-by: Li Wang <liwang@redhat.com>

> ---
>
>  doc/test-writing-guidelines.txt | 10 ++++++++++
>  runtest/staging                 |  1 +
>  2 files changed, 11 insertions(+)
>  create mode 100644 runtest/staging
>
> diff --git a/doc/test-writing-guidelines.txt
> b/doc/test-writing-guidelines.txt
> index 1fa751ed5..8ca014d45 100644
> --- a/doc/test-writing-guidelines.txt
> +++ b/doc/test-writing-guidelines.txt
> @@ -334,3 +334,13 @@ It's easier to maintain a '.gitignore' file per
> directory with tests, rather
>  than having single file in the project root directory. This way, we don't
> have
>  to update all the gitignore files when moving directories, and they get
> deleted
>  automatically when a directory with tests is removed.
> +
> +7 Testing pre-release kernel features
> +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
> +
> +Tests for features not yet in a mainline kernel release are accepted.
> However
> +they must only be added to the +staging+ runtest file. Once a feature is
> part
> +of the stable kernel ABI the associated test must be moved out of staging.
> +
> +This is primarily to help test kernel RCs by avoiding the need to download
> +separate LTP patchsets.
> diff --git a/runtest/staging b/runtest/staging
> new file mode 100644
> index 000000000..ef1cdea15
> --- /dev/null
> +++ b/runtest/staging
> @@ -0,0 +1 @@
> +# Tests for features that are not yet in the stable kernel ABI
> --
> 2.34.1
>
>

Hi all,

...
> > +7 Testing pre-release kernel features
> > +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
> > +
> > +Tests for features not yet in a mainline kernel release are accepted.
> > However
> > +they must only be added to the +staging+ runtest file. Once a feature is
> > part
> > +of the stable kernel ABI the associated test must be moved out of staging.
> > +
> > +This is primarily to help test kernel RCs by avoiding the need to download
> > +separate LTP patchsets.
> > diff --git a/runtest/staging b/runtest/staging
...

FYI keyctl09 [1] patch is a candidate for merging under this policy.

Cyril wasn't sure [2] that we haven't agreed yet about how this should be
handled. Not sure what exactly is he missing.

I guess the description is quite open to allow testing both tests in next tree
or even in maintainer tree (before endup in next tree). From my point it should
be obvious that not *anything* out of mainline can be added, just tests which
sooner or later endup in mainline (and if not, they should be deleted).

Also procedure after release is pretty clear: just move everything out of
staging to expected runtest files (mostly syscalls).

=> I'd merge this.

Kind regards,
Petr

[1] https://patchwork.ozlabs.org/project/ltp/patch/20220223200731.1859670-1-yaelt@google.com/
[2] https://lore.kernel.org/ltp/YiC4Pj1sH8UIHY7k@yuki/

Hi all,

as there is another set of fanotify tests for rc kernel, I'd really like to have
some policy. Therefore I suggest to stick policy suggested by Richie in this
patchset [1], i.e. adding tests into runtest/staging until mainline kernel with
this functionality is released (feature is part of the stable kernel ABI).
i.e. each kernel release content of this file must be revised and (ideally)
moved to particular runtest file where it should belong or fixed (in rare
situations when feature changed) or removed (feature was reverted).

I volunteer to maintain runtest/staging.

This feature got already ack from Mike Frysinger, Li Wang, Jan Stancek (and me).
I'll wait little longer for ack from others (Yang Xu, Cyril Hrubis).

Anybody against it?

"remove_after_release" solution [2] suggested by Cyril and Richie, which would
keep kernel version could be quite easily implemented. But IMHO it's not worth
with current number of tests which need it (< 5, often 1 or 0).

Obviously I suggest to drop my original suggestion [2] (it didn't include
runtest/staging).

Kind regards,
Petr

[1] https://lore.kernel.org/ltp/20220203101803.10204-1-rpalethorpe@suse.com/
[2] https://lore.kernel.org/ltp/YdW5WEXgrotentzM@yuki/
[3] https://lore.kernel.org/ltp/20211210134556.26091-1-pvorel@suse.cz/

Hi!
> as there is another set of fanotify tests for rc kernel, I'd really like to have
> some policy. Therefore I suggest to stick policy suggested by Richie in this
> patchset [1], i.e. adding tests into runtest/staging until mainline kernel with
> this functionality is released (feature is part of the stable kernel ABI).
> i.e. each kernel release content of this file must be revised and (ideally)
> moved to particular runtest file where it should belong or fixed (in rare
> situations when feature changed) or removed (feature was reverted).
> 
> I volunteer to maintain runtest/staging.

As long as you commit to maintain the staging you have my ack as well.

Hi all,

> Hi!
> > as there is another set of fanotify tests for rc kernel, I'd really like to have
> > some policy. Therefore I suggest to stick policy suggested by Richie in this
> > patchset [1], i.e. adding tests into runtest/staging until mainline kernel with
> > this functionality is released (feature is part of the stable kernel ABI).
> > i.e. each kernel release content of this file must be revised and (ideally)
> > moved to particular runtest file where it should belong or fixed (in rare
> > situations when feature changed) or removed (feature was reverted).

> > I volunteer to maintain runtest/staging.

> As long as you commit to maintain the staging you have my ack as well.
Thx! I merged this patch then and I'll merge Amir's fanotify{10,23} tests for
5.19-rc1 kernel.

Kind regards,
Petr