From patchwork Fri May 7 10:32:57 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Joerg Vehlow X-Patchwork-Id: 1475464 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=lists.linux.it (client-ip=2001:1418:10:5::2; helo=picard.linux.it; envelope-from=ltp-bounces+incoming=patchwork.ozlabs.org@lists.linux.it; receiver=) Authentication-Results: ozlabs.org; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=jv-coder.de header.i=@jv-coder.de header.a=rsa-sha256 header.s=dkim header.b=gPWN1p1g; dkim-atps=neutral Received: from picard.linux.it (picard.linux.it [IPv6:2001:1418:10:5::2]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 4Fc6HD3GCdz9sRK for ; Fri, 7 May 2021 20:33:28 +1000 (AEST) Received: from picard.linux.it (localhost [IPv6:::1]) by picard.linux.it (Postfix) with ESMTP id 9227A3C56C1 for ; Fri, 7 May 2021 12:33:21 +0200 (CEST) X-Original-To: ltp@lists.linux.it Delivered-To: ltp@picard.linux.it Received: from in-4.smtp.seeweb.it (in-4.smtp.seeweb.it [IPv6:2001:4b78:1:20::4]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits)) (No client certificate requested) by picard.linux.it (Postfix) with ESMTPS id AD92B3C55FE for ; Fri, 7 May 2021 12:33:10 +0200 (CEST) Received: from mail.jv-coder.de (mail.jv-coder.de [5.9.79.73]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by in-4.smtp.seeweb.it (Postfix) with ESMTPS id 19B4510006B7 for ; Fri, 7 May 2021 12:33:10 +0200 (CEST) Received: from ubuntu.localdomain (unknown [178.26.168.79]) by mail.jv-coder.de (Postfix) with ESMTPSA id 907AC9FBF7; Fri, 7 May 2021 10:33:08 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=jv-coder.de; s=dkim; t=1620383588; bh=/WU5rdGa9NSpB8EJbbtmK7pSJlEdkkZvHXDnPbIysKo=; h=From:To:Subject:Date:Message-Id:MIME-Version; b=gPWN1p1gRHRjH/2yrb/i/wScu0GIn1vViRTswiIV+dEuxhNzuP0OVmwrVppxxNHCA 34VLMi6jjWbXqDnk1Ln5FcM1R7JXlf0MuK6CZ1RBlGj61tbwIlZrLyrv+SC0AFXaM2 OhUqOigxV5E9Hj8wbhCAb2zwAMBoct3bkaHZ56I0= From: Joerg Vehlow To: ltp@lists.linux.it, pvorel@suse.cz, chrubis@suse.cz, liwang@redhat.com Date: Fri, 7 May 2021 12:32:57 +0200 Message-Id: <20210507103258.232174-2-lkml@jv-coder.de> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210507103258.232174-1-lkml@jv-coder.de> References: <20210507103258.232174-1-lkml@jv-coder.de> MIME-Version: 1.0 X-Spam-Status: No, score=0.1 required=7.0 tests=DKIM_SIGNED,DKIM_VALID, DKIM_VALID_AU,DKIM_VALID_EF,SPF_HELO_NONE,SPF_PASS autolearn=disabled version=3.4.4 X-Spam-Checker-Version: SpamAssassin 3.4.4 (2020-01-24) on in-4.smtp.seeweb.it X-Virus-Scanned: clamav-milter 0.102.4 at in-4.smtp.seeweb.it X-Virus-Status: Clean Subject: [LTP] [PATCH v2 1/2] shell: Extend timeout tests X-BeenThere: ltp@lists.linux.it X-Mailman-Version: 2.1.29 Precedence: list List-Id: Linux Test Project List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Joerg Vehlow Errors-To: ltp-bounces+incoming=patchwork.ozlabs.org@lists.linux.it Sender: "ltp" From: Joerg Vehlow - Add test, that verifies, that the timeout is working as expected. - Add test case to check if there are processes left, after a test finished without running into a timeout. - Add execution of timeout03.sh and verification of return code. Signed-off-by: Joerg Vehlow --- lib/newlib_tests/shell/test_timeout.sh | 178 ++++++++++++++++++++++--- lib/newlib_tests/shell/timeout04.sh | 22 +++ 2 files changed, 179 insertions(+), 21 deletions(-) create mode 100755 lib/newlib_tests/shell/timeout04.sh diff --git a/lib/newlib_tests/shell/test_timeout.sh b/lib/newlib_tests/shell/test_timeout.sh index 26477d6f2..06f3acf88 100755 --- a/lib/newlib_tests/shell/test_timeout.sh +++ b/lib/newlib_tests/shell/test_timeout.sh @@ -1,42 +1,178 @@ #!/bin/sh # SPDX-License-Identifier: GPL-2.0-or-later # Copyright (c) 2019 Petr Vorel +# Copyright (c) 2021 Joerg Vehlow PATH="$(dirname $0):$(dirname $0)/../../../testcases/lib/:$PATH" +# Test cases are separated by newlines. +# Every test has the following fields in this order: +# file +# timeout_mul +# use_cat +# max_runtime +# expected_exit_code +# expected passes +# expected failed +# expected broken +# description +# Whole lines can be commented out using "#" DATA=" -timeout01.sh||0 -timeout02.sh||0 -timeout02.sh|foo|2 -timeout02.sh|2|0 -timeout01.sh|2|0 -timeout02.sh|1.1|0 -timeout02.sh|-10|2 -timeout02.sh|-0.1|0 -timeout02.sh|-1.1|2 -timeout02.sh|-10.1|2 +timeout01.sh| |0| |0 +timeout02.sh| |0| |0 +timeout02.sh| foo|0| |2 +timeout02.sh| 2|0| |0 +timeout01.sh| 2|0| |0 +timeout02.sh| 1.1|0| |0 +timeout02.sh| -10|0| |2 +timeout02.sh| -0.1|0| |0 +timeout02.sh| -1.1|0| |2 +timeout02.sh|-10.1|0| |2 +timeout03.sh| |0|12|137| | | |Test kill if test does not terminate by SIGINT +timeout04.sh| |0| | 2|0|0|1|Verify that timeout is enforced +timeout02.sh| 2|1| 2| |1|0|0|Test termination of timeout process " +# Executes a test +# Parameter: +# - test: The test to execute +# - timeout: The timeout multiplicator (optional) +# - use_cat: Pipe the output of the command through cat (optional) +# If this is used, the exit code is NOT returned! +# +# The function returns the following global variables: +# - test_exit: The exit code of the test +# - test_duration: The duration of the test in seconds +# - test_output: The full output of the test +# - test_passed: The number of passed tests parsed from the summary +# - test_failed: The number of failed tests parsed from the summary +# - test_broken: The number of broken tests parsed from the summary +run_test() +{ + local test=$1 + local timeout=$2 + local use_cat=$3 + local tmpfile start end; + + tmpfile=$(mktemp -t ltp_timeout_XXXXXXXX) + start=$(date +%s) + # We have to use set monitor mode (set -m) here. + # In most shells in most cases this creates a + # new process group for the next command. + # A process group is required for the timeout functionality, + # because it sends signals to the whole process group. + set -m + # The use_cat is for testing if any programm using stdout + # is still running, after the test finished. + # cat will wait for its stdin to be closed. + # + # In the pure shell implementation of the timeout handling, + # the sleep process was never killed, when a test finished + # before the timeout. + if [ "$use_cat" = "1" ]; then + LTP_TIMEOUT_MUL=$timeout $test 2>&1 | cat >$tmpfile + else + LTP_TIMEOUT_MUL=$timeout $test 1>$tmpfile 2>&1 + fi + test_exit=$? + set +m + end=$(date +%s) + + test_duration=$((end - start)) + test_output=$(cat $tmpfile) + rm $tmpfile + + eval $(echo "$test_output" | awk ' + BEGIN {sum=0} + $1 == "Summary:" { + sum=1; + } + sum && ( \ + $1 == "passed" \ + || $1 == "failed" \ + || $1 == "broken") { + print "test_" $1 "=" $2 + } + ') +} + echo "Testing timeout in shell API" echo failed=0 -for i in $DATA; do - file=$(echo $i | cut -d'|' -f1) - timeout=$(echo $i | cut -d'|' -f2) - exp_exit=$(echo $i | cut -d'|' -f3) - - echo "=== $file (LTP_TIMEOUT_MUL='$timeout') ===" - LTP_TIMEOUT_MUL=$timeout $file - ret=$? - if [ $ret -ne $exp_exit ]; then - echo "FAILED (exit code: $ret, expected $exp_exit)" - failed=$((failed+1)) +test_nr=0 + +old_ifs="$IFS" +IFS=$(printf "\n\b") + +# Remove comments and empty lines +CLEANED_DATA=$(echo "$DATA" | sed '/^\s*#/d;/^\s*$/d') +test_max=$(echo "$CLEANED_DATA" | wc -l) +for d in $CLEANED_DATA; do + IFS="$old_ifs" + + file=$(echo $d | cut -d'|' -f1 | xargs) + timeout=$(echo $d | cut -d'|' -f2 | xargs) + use_cat=$(echo $d | cut -d'|' -f3 | xargs) + max_runtime=$(echo $d | cut -d'|' -f4 | xargs) + max_runtime=${max_runtime:--1} + exp_exit=$(echo $d | cut -d'|' -f5 | xargs) + exp_exit=${exp_exit:--1} + exp_passed=$(echo $d | cut -d'|' -f6 | xargs) + exp_passed=${exp_passed:--1} + exp_failed=$(echo $d | cut -d'|' -f7 | xargs) + exp_failed=${exp_failed:--1} + exp_broken=$(echo $d | cut -d'|' -f8 | xargs) + exp_broken=${exp_broken:--1} + description=$(echo $d | cut -d'|' -f9) + + test_nr=$(($test_nr + 1)) + + cur_fails=0 + + if [ -z "$description" ]; then + description="$file (LTP_TIMEOUT_MUL='$timeout')" + fi + + echo "=== $test_nr/$test_max $description ===" + run_test "$file" "$timeout" "$use_cat" + + if [ $max_runtime -ne -1 ] && [ $test_duration -gt $max_runtime ]; then + echo "FAILED (runtime: $test_duration, expected less than $max_runtime)" + cur_fails=$((cur_fails + 1)) + fi + + if [ $exp_passed -ne -1 ] && [ $exp_passed -ne $test_passed ]; then + echo "FAILED (passes: $test_passed, expected $exp_passed)" + cur_fails=$((cur_fails + 1)) + fi + + if [ $exp_failed -ne -1 ] && [ $exp_failed -ne $test_failed ]; then + echo "FAILED (failed: $test_failed, expected $exp_failed)" + cur_fails=$((cur_fails + 1)) + fi + + if [ $exp_broken -ne -1 ] && [ $exp_broken -ne $test_broken ]; then + echo "FAILED (broken: $test_broken, expected $exp_broken)" + cur_fails=$((cur_fails + 1)) + fi + + if [ $exp_exit -ne -1 ] && [ $test_exit -ne $exp_exit ]; then + echo "FAILED (exit code: $test_exit, expected $exp_exit)" + cur_fails=$((cur_fails + 1)) + fi + + if [ $cur_fails -gt 0 ]; then + failed=$((failed + 1)) + echo "--------" + echo "$test_output" + echo "--------" else echo "PASSED" fi echo done +IFS="$old_ifs" echo "Failed tests: $failed" exit $failed diff --git a/lib/newlib_tests/shell/timeout04.sh b/lib/newlib_tests/shell/timeout04.sh new file mode 100755 index 000000000..0a6ba053c --- /dev/null +++ b/lib/newlib_tests/shell/timeout04.sh @@ -0,0 +1,22 @@ +#!/bin/sh +# SPDX-License-Identifier: GPL-2.0-or-later +# Copyright (c) 2021 Joerg Vehlow + +TST_TESTFUNC=do_test + +TST_TIMEOUT=1 +. tst_test.sh + +do_test() +{ + tst_res TINFO "Start" + sleep 5 + tst_res TFAIL "End" +} + +do_cleanup() +{ + tst_res TINFO "cleanup" +} + +tst_run From patchwork Fri May 7 10:32:58 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Joerg Vehlow X-Patchwork-Id: 1475465 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=lists.linux.it (client-ip=2001:1418:10:5::2; helo=picard.linux.it; envelope-from=ltp-bounces+incoming=patchwork.ozlabs.org@lists.linux.it; receiver=) Authentication-Results: ozlabs.org; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=jv-coder.de header.i=@jv-coder.de header.a=rsa-sha256 header.s=dkim header.b=pOnt7nw2; dkim-atps=neutral Received: from picard.linux.it (picard.linux.it [IPv6:2001:1418:10:5::2]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 4Fc6HK4WHQz9sRK for ; Fri, 7 May 2021 20:33:33 +1000 (AEST) Received: from picard.linux.it (localhost [IPv6:::1]) by picard.linux.it (Postfix) with ESMTP id 1EADD3C564D for ; Fri, 7 May 2021 12:33:31 +0200 (CEST) X-Original-To: ltp@lists.linux.it Delivered-To: ltp@picard.linux.it Received: from in-4.smtp.seeweb.it (in-4.smtp.seeweb.it [217.194.8.4]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits)) (No client certificate requested) by picard.linux.it (Postfix) with ESMTPS id 843633C7009 for ; Fri, 7 May 2021 12:33:11 +0200 (CEST) Received: from mail.jv-coder.de (mail.jv-coder.de [5.9.79.73]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by in-4.smtp.seeweb.it (Postfix) with ESMTPS id EA7D110001A1 for ; Fri, 7 May 2021 12:33:10 +0200 (CEST) Received: from ubuntu.localdomain (unknown [178.26.168.79]) by mail.jv-coder.de (Postfix) with ESMTPSA id C93ED9FBF9; Fri, 7 May 2021 10:33:09 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=jv-coder.de; s=dkim; t=1620383590; bh=QYPfNfWg+qg83Ub72oxqCpqIqlFkyAY/+xl7LgFeat0=; h=From:To:Subject:Date:Message-Id:MIME-Version; b=pOnt7nw2mZUnI4knyv2Vz44+hOOSodgvQ0kJGZtRg0CY80n0mV9BLlyQlZscCaDZa rlbnR3c9AKRa9qEmTWehZyV1UYtAeMoaIxB38okwE/GDwe/JiEMAhI4drygHhXoxqd 5k9TaXU/wRh+NfRtArCHKPGH9IqSdAPym0bc+jUk= From: Joerg Vehlow To: ltp@lists.linux.it, pvorel@suse.cz, chrubis@suse.cz, liwang@redhat.com Date: Fri, 7 May 2021 12:32:58 +0200 Message-Id: <20210507103258.232174-3-lkml@jv-coder.de> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210507103258.232174-1-lkml@jv-coder.de> References: <20210507103258.232174-1-lkml@jv-coder.de> MIME-Version: 1.0 X-Spam-Status: No, score=0.1 required=7.0 tests=DKIM_SIGNED,DKIM_VALID, DKIM_VALID_AU,DKIM_VALID_EF,SPF_HELO_NONE,SPF_PASS autolearn=disabled version=3.4.4 X-Spam-Checker-Version: SpamAssassin 3.4.4 (2020-01-24) on in-4.smtp.seeweb.it X-Virus-Scanned: clamav-milter 0.102.4 at in-4.smtp.seeweb.it X-Virus-Status: Clean Subject: [LTP] [PATCH v2 2/2] shell: Prevent orphan timeout sleep processes X-BeenThere: ltp@lists.linux.it X-Mailman-Version: 2.1.29 Precedence: list List-Id: Linux Test Project List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Joerg Vehlow Errors-To: ltp-bounces+incoming=patchwork.ozlabs.org@lists.linux.it Sender: "ltp" From: Joerg Vehlow The implementation of the timeout handling failed to terminate the sleep process, because the command "sleep $sec && _tst_kill_test &" spawns two processes: A shell, that is used to execute the sleep and after sleep terminates it executes _tst_kill_test. The pid stored in $! is the pid of the shell process. When the test finished, the timeout process is supposed to be killed, but only the shell process is killed, not the sleep. If the output of the test is piped somewhere else, the pipe stays open, until the sleep finished, because it still has stdout open. Additionally, if a lot of short tests with high timeouts are executed, a lot of sleep processes will linger around. Co-authored-by: Li Wang Signed-off-by: Joerg Vehlow Reviewed-by: Li Wang Acked-by: Petr Vorel --- testcases/lib/tst_test.sh | 30 +++++++++++++++++++++--------- 1 file changed, 21 insertions(+), 9 deletions(-) diff --git a/testcases/lib/tst_test.sh b/testcases/lib/tst_test.sh index 951518785..7ceddff04 100644 --- a/testcases/lib/tst_test.sh +++ b/testcases/lib/tst_test.sh @@ -23,14 +23,6 @@ export TST_LIB_LOADED=1 # default trap function trap "tst_brk TBROK 'test interrupted or timed out'" INT -_tst_cleanup_timer() -{ - if [ -n "$_tst_setup_timer_pid" ]; then - kill $_tst_setup_timer_pid 2>/dev/null - wait $_tst_setup_timer_pid 2>/dev/null - fi -} - _tst_do_exit() { local ret=0 @@ -463,6 +455,25 @@ _tst_kill_test() fi } +_tst_cleanup_timer() +{ + if [ -n "$_tst_setup_timer_pid" ]; then + kill -TERM $_tst_setup_timer_pid 2>/dev/null + wait $_tst_setup_timer_pid 2>/dev/null + fi +} + +_tst_timeout_process() +{ + local sleep_pid + + sleep $sec & + sleep_pid=$! + #trap "kill $sleep_pid; exit" TERM + wait $sleep_pid + _tst_kill_test +} + _tst_setup_timer() { TST_TIMEOUT=${TST_TIMEOUT:-300} @@ -486,7 +497,8 @@ _tst_setup_timer() tst_res TINFO "timeout per run is ${h}h ${m}m ${s}s" _tst_cleanup_timer - sleep $sec && _tst_kill_test & + + _tst_timeout_process & _tst_setup_timer_pid=$! }