diff mbox series

[01/10] Add the new basics documentation

Message ID 20240320-new_website-v1-1-79b603c8aea1@suse.com
State Superseded
Headers show
Series New LTP documentation | expand

Commit Message

Andrea Cervesato March 20, 2024, 4:20 p.m. UTC
From: Andrea Cervesato <andrea.cervesato@suse.com>

This is the first patch introducing the new LTP documentation. The
new documentation is now using Sphinx as the main framework,
providing a solid substitute to Github wiki pages.

This patch is meant to be the starting point, where we start rewriting
the old asciidoc documentation format.

A .readthedocs.yaml configuration has been added to provide an initial
documentation publish via readthedocs service.
---
 .readthedocs.yml                              |  15 +++
 doc_new/.gitignore                            |   1 +
 doc_new/Makefile                              |   5 +
 doc_new/_static/custom.css                    |   4 +
 doc_new/conf.py                               |  27 ++++++
 doc_new/developers/api_c_tests.rst            |   4 +
 doc_new/developers/api_kvm_tests.rst          |   4 +
 doc_new/developers/api_network_tests.rst      |   4 +
 doc_new/developers/api_shell_tests.rst        |   4 +
 doc_new/developers/build_system.rst           |   4 +
 doc_new/developers/debugging.rst              |  10 ++
 doc_new/developers/ltp_library.rst            |   4 +
 doc_new/developers/setup_mailinglist.rst      |   4 +
 doc_new/developers/test_case_tutorial.rst     |   4 +
 doc_new/developers/writing_tests.rst          |  20 ++++
 doc_new/index.rst                             | 124 +++++++++++++++++++++++++
 doc_new/maintainers/ltp_release_procedure.rst |   4 +
 doc_new/maintainers/patch_review.rst          |   4 +
 doc_new/requirements.txt                      |   1 +
 doc_new/users/quick_start.rst                 | 126 ++++++++++++++++++++++++++
 doc_new/users/setup_tests.rst                 | 104 +++++++++++++++++++++
 doc_new/users/supported_systems.rst           | 110 ++++++++++++++++++++++
 22 files changed, 587 insertions(+)
diff mbox series

Patch

diff --git a/.readthedocs.yml b/.readthedocs.yml
new file mode 100644
index 000000000..636e29c97
--- /dev/null
+++ b/.readthedocs.yml
@@ -0,0 +1,15 @@ 
+version: 2
+
+build:
+  os: "ubuntu-22.04"
+  tools:
+    python: "3.6"
+
+# Build from the docs/ directory with Sphinx
+sphinx:
+  configuration: doc_new/conf.py
+
+# Explicitly set the version of Python and its requirements
+python:
+  install:
+    - requirements: doc_new/requirements.txt
diff --git a/doc_new/.gitignore b/doc_new/.gitignore
new file mode 100644
index 000000000..5ccff1a6b
--- /dev/null
+++ b/doc_new/.gitignore
@@ -0,0 +1 @@ 
+html/
diff --git a/doc_new/Makefile b/doc_new/Makefile
new file mode 100644
index 000000000..aea89f641
--- /dev/null
+++ b/doc_new/Makefile
@@ -0,0 +1,5 @@ 
+all:
+	sphinx-build -b html . html
+
+clean:
+	rm -rf html/
diff --git a/doc_new/_static/custom.css b/doc_new/_static/custom.css
new file mode 100644
index 000000000..cb4d3821d
--- /dev/null
+++ b/doc_new/_static/custom.css
@@ -0,0 +1,4 @@ 
+/* set multiline tables cells */
+.wy-table-responsive table td {
+    white-space: normal;
+}
diff --git a/doc_new/conf.py b/doc_new/conf.py
new file mode 100644
index 000000000..b333ebf2a
--- /dev/null
+++ b/doc_new/conf.py
@@ -0,0 +1,27 @@ 
+# Configuration file for the Sphinx documentation builder.
+#
+# For the full list of built-in configuration values, see the documentation:
+# https://www.sphinx-doc.org/en/master/usage/configuration.html
+
+# -- Project information -----------------------------------------------------
+# https://www.sphinx-doc.org/en/master/usage/configuration.html#project-information
+
+project = 'Linux Test Project'
+copyright = '2024, Linux Test Project'
+author = 'Linux Test Project'
+release = '1.0'
+
+# -- General configuration ---------------------------------------------------
+# https://www.sphinx-doc.org/en/master/usage/configuration.html#general-configuration
+
+extensions = []
+
+
+# -- Options for HTML output -------------------------------------------------
+# https://www.sphinx-doc.org/en/master/usage/configuration.html#options-for-html-output
+
+html_theme = 'sphinx_rtd_theme'
+html_static_path = ['_static']
+
+def setup(app):
+    app.add_css_file('custom.css')
diff --git a/doc_new/developers/api_c_tests.rst b/doc_new/developers/api_c_tests.rst
new file mode 100644
index 000000000..c03f9caf1
--- /dev/null
+++ b/doc_new/developers/api_c_tests.rst
@@ -0,0 +1,4 @@ 
+.. SPDX-License-Identifier: GPL-2.0-or-later
+
+Developing using C API
+======================
diff --git a/doc_new/developers/api_kvm_tests.rst b/doc_new/developers/api_kvm_tests.rst
new file mode 100644
index 000000000..1ef7ec452
--- /dev/null
+++ b/doc_new/developers/api_kvm_tests.rst
@@ -0,0 +1,4 @@ 
+.. SPDX-License-Identifier: GPL-2.0-or-later
+
+Developing using KVM API
+========================
diff --git a/doc_new/developers/api_network_tests.rst b/doc_new/developers/api_network_tests.rst
new file mode 100644
index 000000000..4fe45f5f6
--- /dev/null
+++ b/doc_new/developers/api_network_tests.rst
@@ -0,0 +1,4 @@ 
+.. SPDX-License-Identifier: GPL-2.0-or-later
+
+Developing using network API
+============================
diff --git a/doc_new/developers/api_shell_tests.rst b/doc_new/developers/api_shell_tests.rst
new file mode 100644
index 000000000..2fb2a5068
--- /dev/null
+++ b/doc_new/developers/api_shell_tests.rst
@@ -0,0 +1,4 @@ 
+.. SPDX-License-Identifier: GPL-2.0-or-later
+
+Developing using shell API
+==========================
diff --git a/doc_new/developers/build_system.rst b/doc_new/developers/build_system.rst
new file mode 100644
index 000000000..cfcaf2d04
--- /dev/null
+++ b/doc_new/developers/build_system.rst
@@ -0,0 +1,4 @@ 
+.. SPDX-License-Identifier: GPL-2.0-or-later
+
+Build system
+============
diff --git a/doc_new/developers/debugging.rst b/doc_new/developers/debugging.rst
new file mode 100644
index 000000000..18a4a67e9
--- /dev/null
+++ b/doc_new/developers/debugging.rst
@@ -0,0 +1,10 @@ 
+.. SPDX-License-Identifier: GPL-2.0-or-later
+
+Debugging
+=========
+
+The new test library runs the actual test, i.e. the `test()` function in a
+forked process. To get stack trace of a crashing test in gdb it's needed to
+`set follow-fork-mode child <https://ftp.gnu.org/old-gnu/Manuals/gdb/html_node/gdb_25.html>`_.
+To trace the test, please use `strace -f` to enable tracing also for the
+forked processes.
diff --git a/doc_new/developers/ltp_library.rst b/doc_new/developers/ltp_library.rst
new file mode 100644
index 000000000..b780b6870
--- /dev/null
+++ b/doc_new/developers/ltp_library.rst
@@ -0,0 +1,4 @@ 
+.. SPDX-License-Identifier: GPL-2.0-or-later
+
+LTP Library development
+=======================
diff --git a/doc_new/developers/setup_mailinglist.rst b/doc_new/developers/setup_mailinglist.rst
new file mode 100644
index 000000000..b7320b9a2
--- /dev/null
+++ b/doc_new/developers/setup_mailinglist.rst
@@ -0,0 +1,4 @@ 
+.. SPDX-License-Identifier: GPL-2.0-or-later
+
+Setting up the Mailing list
+===========================
diff --git a/doc_new/developers/test_case_tutorial.rst b/doc_new/developers/test_case_tutorial.rst
new file mode 100644
index 000000000..43563c234
--- /dev/null
+++ b/doc_new/developers/test_case_tutorial.rst
@@ -0,0 +1,4 @@ 
+.. SPDX-License-Identifier: GPL-2.0-or-later
+
+Test case tutorial
+==================
diff --git a/doc_new/developers/writing_tests.rst b/doc_new/developers/writing_tests.rst
new file mode 100644
index 000000000..ba0188aee
--- /dev/null
+++ b/doc_new/developers/writing_tests.rst
@@ -0,0 +1,20 @@ 
+.. SPDX-License-Identifier: GPL-2.0-or-later
+
+Writing tests
+=============
+
+Testing builds with GitHub Actions
+----------------------------------
+
+Master branch is tested in `GitHub Actions <https://github.com/linux-test-project/ltp/actions>`_
+to ensure LTP builds in various distributions, including old, current and
+bleeding edge. ``gcc`` and ``clang`` toolchains are also tested for various
+architectures using cross-compilation. For a full list of tested distros, please
+check ``.github/workflows/ci.yml``.
+
+.. note::
+
+      Passing the GitHub Actions CI means that LTP compiles in a variety of
+      different distributions on their **newest releases**.
+      The CI also checks for code linting, running ``make check`` in the whole
+      LTP project.
diff --git a/doc_new/index.rst b/doc_new/index.rst
new file mode 100644
index 000000000..a8778b3d8
--- /dev/null
+++ b/doc_new/index.rst
@@ -0,0 +1,124 @@ 
+.. SPDX-License-Identifier: GPL-2.0-or-later
+
+Linux Test Project
+==================
+
+Linux Test Project is a joint project started by SGI, OSDL and Bull developed
+and maintained by SUSE, Red Hat, Fujitsu, IBM, Cisco, Oracle and others. The
+project goal is to deliver tests to the open source community that validate
+reliability, robustness, and stability of the Linux Kernel.
+
+The testing suites contain a collection of tools for testing the Linux kernel
+and related features. Our goal is to improve the Linux kernel and system
+libraries by bringing test automation.
+
+.. warning::
+
+   LTP tests shouldn't run in production systems. In particular,
+   growfiles, doio, and iogen, stress the I/O capabilities of the systems and
+   they are intended to find (or cause) problems.
+
+Some references:
+
+* `Source code <https://github.com/linux-test-project/ltp>`_
+* `Releases <https://github.com/linux-test-project/ltp/releases>`_
+* `Mailing List <http://lists.linux.it/listinfo/ltp>`_
+* `Working patches (patchwork) <https://patchwork.ozlabs.org/project/ltp/list/>`_
+* `Working patches (lore.kernel.org) <https://lore.kernel.org/ltp>`_
+* `#ltp @ libera chat <https://libera.chat/>`_
+
+.. toctree::
+   :maxdepth: 3
+   :hidden:
+   :caption: For users
+
+   users/quick_start
+   users/setup_tests
+   users/supported_systems
+
+.. toctree::
+   :maxdepth: 3
+   :hidden:
+   :caption: For developers
+
+   developers/setup_mailinglist
+   developers/writing_tests
+   developers/test_case_tutorial
+   developers/api_c_tests
+   developers/api_shell_tests
+   developers/api_network_tests
+   developers/api_kvm_tests
+   developers/ltp_library
+   developers/build_system
+   developers/debugging
+
+.. toctree::
+   :maxdepth: 3
+   :hidden:
+   :caption: For maintainers
+
+   maintainers/patch_review
+   maintainers/ltp_release_procedure
+
+For users
+---------
+
+.. descriptions here are active
+
+:doc:`users/quick_start`
+   How to build and use LTP framework in few steps
+
+:doc:`users/setup_tests`
+   How to setup tests execution
+
+:doc:`users/supported_systems`
+   A list of supported technologies by the LTP framework
+
+For developers
+--------------
+
+.. descriptions here are active
+
+:doc:`developers/setup_mailinglist`
+   How to configure git and to start sending patches via ``git send-email``
+
+:doc:`developers/writing_tests`
+   Starting guide on writing tests
+
+:doc:`developers/test_case_tutorial`
+   A tutorial showing how to write a test from scratch using C API
+
+:doc:`developers/api_c_tests`
+   Walk through the C API features
+
+:doc:`developers/api_shell_tests`
+   Walk through the Shell API features
+
+:doc:`developers/api_network_tests`
+   Walk through the network API features
+
+:doc:`developers/api_kvm_tests`
+   Walk through the KVM API features
+
+:doc:`developers/ltp_library`
+   Developing new features in the LTP library
+
+:doc:`developers/build_system`
+   Short introduction to the LTP build system
+
+:doc:`developers/debugging`
+   How to debug LTP tests
+
+For maintainers
+---------------
+
+:doc:`maintainers/patch_review`
+   Steps to follow when reviewing patches
+
+:doc:`maintainers/ltp_release_procedure`
+   Steps to follow for a new LTP release
+
+
+Getting help
+------------
+To report a problem or suggest any feature, please write at ltp@lists.linux.it
diff --git a/doc_new/maintainers/ltp_release_procedure.rst b/doc_new/maintainers/ltp_release_procedure.rst
new file mode 100644
index 000000000..808ff2211
--- /dev/null
+++ b/doc_new/maintainers/ltp_release_procedure.rst
@@ -0,0 +1,4 @@ 
+.. SPDX-License-Identifier: GPL-2.0-or-later
+
+Release process
+===============
diff --git a/doc_new/maintainers/patch_review.rst b/doc_new/maintainers/patch_review.rst
new file mode 100644
index 000000000..1fb76b8bc
--- /dev/null
+++ b/doc_new/maintainers/patch_review.rst
@@ -0,0 +1,4 @@ 
+.. SPDX-License-Identifier: GPL-2.0-or-later
+
+Patch reviewing
+===============
diff --git a/doc_new/requirements.txt b/doc_new/requirements.txt
new file mode 100644
index 000000000..59709ec72
--- /dev/null
+++ b/doc_new/requirements.txt
@@ -0,0 +1 @@ 
+sphinx-rtd-theme==2.0.0
diff --git a/doc_new/users/quick_start.rst b/doc_new/users/quick_start.rst
new file mode 100644
index 000000000..40d8dd9c6
--- /dev/null
+++ b/doc_new/users/quick_start.rst
@@ -0,0 +1,126 @@ 
+.. SPDX-License-Identifier: GPL-2.0-or-later
+
+Installation and tests execution
+================================
+
+Basics requirements to build LTP are the following:
+
+* git
+* autoconf
+* automake
+* make
+* gcc
+* m4
+* pkgconf / pkg-config
+* libc headers
+* linux headers
+
+.. code-block:: console
+
+   $ git clone https://github.com/linux-test-project/ltp.git
+   $ cd ltp
+   $ make autotools
+   $ ./configure
+
+.. note::
+
+   For optional library dependencies, take a look at the scripts inside ``ci/``
+   directory.
+
+Running single tests
+--------------------
+
+LTP provides the possibility to build and run single tests:
+
+.. code-block:: console
+
+   $ cd testcases/kernel/syscalls/foo
+   $ make
+   $ PATH=$PATH:$PWD ./foo01
+
+Shell testcases are a bit more complicated, since they need to setup ``PATH``
+as well as to compiled binary helpers:
+
+.. code-block:: console
+
+   $ cd testcases/lib
+   $ make
+   $ cd ../commands/foo
+   $ PATH=$PATH:$PWD:$PWD/../../lib/ ./foo01.sh
+
+Open Posix Testsuite has it's own build system which needs Makefiles to be
+generated first:
+
+.. code-block:: console
+
+   $ cd testcases/open_posix_testsuite/
+   $ make generate-makefiles
+   $ cd conformance/interfaces/foo
+   $ make
+   $ ./foo_1-1.run-test
+
+Compiling and installing all testcases
+--------------------------------------
+
+To compile all tests is really simple:
+
+.. code-block:: console
+
+   $ make
+
+   $ # install LTP inside /opt/ltp by default
+   $ make install
+
+.. note::
+
+   Some tests will be disabled if ``configure`` script won't find the build
+   dependencies.
+
+Running tests
+-------------
+
+To run all the test suites
+
+.. code-block:: console
+
+   $ cd /opt/ltp
+
+   $ # run syscalls testing suite
+   $ ./kirk -f ltp -r syscalls
+
+.. note::
+
+   Many test cases have to be executed as root.
+
+Test suites (e.g. syscalls) are defined in the ``runtest`` directory. Each file
+contains a list of test cases in a simple format.
+
+Each test case has its own executable or script that can directly executed:
+
+.. code-block:: console
+
+   $ testcases/bin/abort01
+
+   $ # some tests have arguments
+   $ testcases/bin/mesgq_nstest -m none
+
+   $ # vast majority of tests have a help
+   $ testcases/bin/ioctl01 -h
+
+   $ # Many require certain environment variables to be set
+   $ LTPROOT=/opt/ltp PATH="$PATH:$LTPROOT/testcases/bin" testcases/bin/wc01.sh
+
+Most commonly, the ``PATH`` variable needs to be set and also ``LTPROOT``, but
+there are a number of other variables which usually ``kirk`` sets for you.
+
+.. note::
+
+   All shell scripts need the ``PATH`` to be set. However, this is not limited
+   to shell scripts and some C based tests need environment variables as well.
+   They usually raise a configuration error when this is needed.
+
+Network tests
+-------------
+
+Network tests usually require a certain setup that is described in
+``testcases/network/README.md``.
diff --git a/doc_new/users/setup_tests.rst b/doc_new/users/setup_tests.rst
new file mode 100644
index 000000000..910cc2315
--- /dev/null
+++ b/doc_new/users/setup_tests.rst
@@ -0,0 +1,104 @@ 
+.. SPDX-License-Identifier: GPL-2.0-or-later
+
+Tests setup
+===========
+
+The internal LTP library provides a set of features that permits to customize
+tests behaviour by setting environment variables and using specific tests
+arguments.
+
+Library environment variables
+-----------------------------
+
+Following environment variables are expected to be set by LTP users. Therefore,
+with some exceptions, they have ``LTP_`` prefix. Environment variables with
+``TST_`` prefix are used inside LTP shell API and should **not** be set by
+users.
+
+.. list-table::
+   :header-rows: 1
+
+   * - Variable
+     - Note
+
+   * - KCONFIG_PATH
+     - The path to the kernel config file, (if not set, it tries the usual paths
+       ``/boot/config-RELEASE`` or ``/proc/config.gz``)
+
+   * - KCONFIG_SKIP_CHECK
+     - Skip kernel config check if variable set (not set by default)
+
+   * - LTPROOT
+     - Prefix for installed LTP.  **Should be always set**, since some tests
+       need it to use data files (``LTP_DATAROOT``). LTP is by default installed
+       into ``/opt/ltp``
+
+   * - LTP_COLORIZE_OUTPUT
+     - By default LTP colorizes it's output unless it's redirected to a pipe or
+       file. Force colorized output behaviour: ``y`` or ``1``: always colorize,
+       ``n`` or ``0``: never colorize.
+
+   * - LTP_DEV
+     - Path to the block device to be used. C Language: ``.needs_device = 1``.
+       Shell language: ``TST_NEEDS_DEVICE=1``.
+
+   * - LTP_SINGLE_FS_TYPE
+     - Testing only - specifies filesystem instead all supported
+       (for tests with ``.all_filesystems``).
+
+   * - LTP_DEV_FS_TYPE
+     - Filesystem used for testing (default: ``ext2``).
+
+   * - LTP_TIMEOUT_MUL
+     - Multiplies timeout, must be number >= 0.1 (> 1 is useful for slow
+       machines to avoid unexpected timeout).
+
+   * - LTP_RUNTIME_MUL
+     - Multiplies maximal test iteration runtime. Tests that run for more than a
+       second or two are capped on runtime. You can scale the default runtime
+       both up and down with this multiplier. This is not yet implemented in the
+       shell API.
+
+   * - LTP_VIRT_OVERRIDE
+     - Overrides virtual machine detection in the test library. Setting it to
+       empty string, tella the library that system is not a virtual machine.
+       Other possible values are ``kvm``, ``xen``, ``zvm`` and ``microsoft``
+       that describe different types supervisors.
+
+   * - PATH
+     - It's required to addjust path: ``PATH="$PATH:$LTPROOT/testcases/bin"``
+
+   * - TMPDIR
+     - Base directory for template directory (C language: ``.needs_tmpdir = 1``
+       and shell: ``TST_NEEDS_TMPDIR=1``). Must be an absolute path (default:
+       '/tmp').
+
+   * - LTP_NO_CLEANUP
+     - Disable running test cleanup (defined in ``TST_CLEANUP``).
+       Shell API only.
+
+   * - LTP_ENABLE_DEBUG
+     - Enable debug info (value ``1`` or ``y``).
+
+Test execution time and timeout
+-------------------------------
+
+The limit on how long a test can run does compose of two parts: ``max_runtime``
+and ``timeout``. The limit does apply to a single test variant. That means, for
+example, that tests which run for all available filesystems will apply this
+limit for a single filesystem only.
+
+The ``max_runtime`` is a cap on how long the ``run()`` function can take and for
+most testcases this part is set to zero. For tests that do run for more than a
+second or two the ``max_runtime`` has to be defined and the ``run()`` function
+has to check actively how much runtime is left.
+
+Test runtime can be scaled up and down with ``LTP_RUNTIME_MUL`` environment
+variable or set on a commandline by the ``-I`` parameter. Hoewever,
+setting the runtime too low will cause long running tests to exit prematurely,
+possibly before having a chance to actually test anything.
+
+The timeout is a limit for test setup and cleanup and it's also a safety
+margin for the runtime accounting. It's currently set to 30 seconds but it may
+change later. If your target machine is too slow, it can be scaled up with the
+``LTP_TIMEOUT_MUL`` environment variable.
diff --git a/doc_new/users/supported_systems.rst b/doc_new/users/supported_systems.rst
new file mode 100644
index 000000000..6220a96e0
--- /dev/null
+++ b/doc_new/users/supported_systems.rst
@@ -0,0 +1,110 @@ 
+.. SPDX-License-Identifier: GPL-2.0-or-later
+
+Supported systems
+=================
+
+Kernel version
+--------------
+
+Minimal supported kernel version is **3.10**.
+
+Oldest tested distributions
+---------------------------
+
+.. list-table::
+    :header-rows: 1
+
+    * - Distro
+      - Kernel
+      - glibc
+      - gcc
+      - clang
+
+    * - openSUSE Leap 42.2
+      - 4.4
+      - 2.22
+      - 4.8.5
+      - \-
+
+    * - Ubuntu 18.04 LTS bionic
+      - 4.15
+      - 2.27
+      - 7.3.0
+      - \-
+
+    * - Debian 11 (bullseye)
+      - 5.10
+      - 2.31
+      - 10.2.1
+      - 11.0.1
+
+    * - RHEL-7.9 Maipo
+      - 3.10
+      - 4.8.5
+      - 2.17
+      - \-
+
+Older distributions are not officially supported, which means that it
+may or may not work. It all depends on your luck. It should be possible
+to compile latest LTP even on slightly older distributions than we
+support with a few manual tweaks, e.g. disabling manually tests for
+newly added syscalls, etc. **Trivial fixes/workarounds may be accepted,
+but users are encouraged to move to a newer distro.**
+
+If latest LTP cannot be compiled even with some amount of workarounds,
+you may result to older LTP releases, however these are **not** supported
+in any way. Also if you are trying to run LTP on more than 10 years old
+distribution you may as well reconsider you life choices.
+
+Tested architectures
+--------------------
+
+.. list-table::
+    :header-rows: 1
+
+    * - Architecture
+      - Build
+
+    * - x86_64
+      - native
+
+    * - x86 emulation
+      - native
+
+    * - aarch64
+      - cross compilation
+
+    * - ppc64le
+      - cross compilation
+
+    * - s390x
+      - cross compilation
+
+Supported C libraries
+---------------------
+
+.. list-table::
+    :header-rows: 1
+
+    * - C library
+      - Note
+
+    * - `glibc <https://www.gnu.org/software/libc/>`_
+      - Targeted libc, tested both compilation and actual test results.
+
+    * - `uClibc-ng <https://uclibc-ng.org/>`_
+      - Although not being tested, it should work as it attempt to maintain a glibc compatible interface.
+
+    * - `uClibc <https://www.uclibc.org/>`_
+      - Older uClibc might have problems.
+
+    * - `musl <https://musl.libc.org/>`_
+      - Not yet fully supported. Check ``ci/alpine.sh`` script.
+
+    * - Android
+      - Please use `AOSP fork <https://android.googlesource.com/platform/external/ltp>`_
+
+C version
+---------
+
+LTP is compiled with ``-std=gnu99``.