niansa-misc/linux
1
0
Fork 0
mirror of synced 2025-03-06 20:59:54 +01:00
Code Issues Projects Releases Packages Wiki Activity

selftests: mptcp: lib: format subtests results in TAP

Browse source 
The current selftests infrastructure formats the results in TAP 13. This
version doesn't support subtests and only the end result of each
selftest is taken into account. It means that a single issue in a
subtest of a selftest containing multiple subtests forces the whole
selftest to be marked as failed. It also means that subtests results are
not tracked by CIs executing selftests.

MPTCP selftests run hundreds of various subtests. It is then important
to track each of them and not one result per selftest.

It is particularly interesting to do that when validating stable kernels
with the last version of the test suite: tests might fail because a
feature is not supported but the test didn't skip that part. In this
case, if subtests are not tracked, the whole selftest will be marked as
failed making the other subtests useless because their results are
ignored.

This patch adds some helpers in mptcp_lib.sh to be able to easily format
subtests results in TAP in the different MPTCP selftests.

Closes: https://github.com/multipath-tcp/mptcp_net-next/issues/368
Acked-by: Paolo Abeni <pabeni@redhat.com>
Signed-off-by: Matthieu Baerts <matthieu.baerts@tessares.net>
Signed-off-by: David S. Miller <davem@davemloft.net>
This commit is contained in:
Matthieu Baerts 2023-07-17 15:21:26 +02:00 • committed by David S. Miller
parent d8463d8165
commit c4192967e6
1 changed files with 66 additions and 0 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

66
tools/testing/selftests/net/mptcp/mptcp_lib.sh
View file

@ -1,8 +1,12 @@
#! /bin/bash #! /bin/bash
# SPDX-License-Identifier: GPL-2.0 # SPDX-License-Identifier: GPL-2.0
readonly KSFT_PASS=0
readonly KSFT_FAIL=1 readonly KSFT_FAIL=1
readonly KSFT_SKIP=4 readonly KSFT_SKIP=4
readonly KSFT_TEST=$(basename "${0}" | sed 's/\.sh$//g')
MPTCP_LIB_SUBTESTS=()
# SELFTESTS_MPTCP_LIB_EXPECT_ALL_FEATURES env var can be set when validating all # SELFTESTS_MPTCP_LIB_EXPECT_ALL_FEATURES env var can be set when validating all
# features using the last version of the kernel and the selftests to make sure # features using the last version of the kernel and the selftests to make sure
@ -102,3 +106,65 @@ mptcp_lib_kversion_ge() {
mptcp_lib_fail_if_expected_feature "kernel version ${1} lower than ${v}" mptcp_lib_fail_if_expected_feature "kernel version ${1} lower than ${v}"
} }
__mptcp_lib_result_add() {
local result="${1}"
shift
local id=$((${#MPTCP_LIB_SUBTESTS[@]} + 1))
MPTCP_LIB_SUBTESTS+=("${result} ${id} - ${KSFT_TEST}: ${*}")
}
# $1: test name
mptcp_lib_result_pass() {
__mptcp_lib_result_add "ok" "${1}"
}
# $1: test name
mptcp_lib_result_fail() {
__mptcp_lib_result_add "not ok" "${1}"
}
# $1: test name
mptcp_lib_result_skip() {
__mptcp_lib_result_add "ok" "${1} # SKIP"
}
# $1: result code ; $2: test name
mptcp_lib_result_code() {
local ret="${1}"
local name="${2}"
case "${ret}" in
"${KSFT_PASS}")
mptcp_lib_result_pass "${name}"
;;
"${KSFT_FAIL}")
mptcp_lib_result_fail "${name}"
;;
"${KSFT_SKIP}")
mptcp_lib_result_skip "${name}"
;;
*)
echo "ERROR: wrong result code: ${ret}"
exit ${KSFT_FAIL}
;;
esac
}
mptcp_lib_result_print_all_tap() {
local subtest
if [ ${#MPTCP_LIB_SUBTESTS[@]} -eq 0 ] ||
[ "${SELFTESTS_MPTCP_LIB_NO_TAP:-}" = "1" ]; then
return
fi
printf "\nTAP version 13\n"
printf "1..%d\n" "${#MPTCP_LIB_SUBTESTS[@]}"
for subtest in "${MPTCP_LIB_SUBTESTS[@]}"; do
printf "%s\n" "${subtest}"
done
}