Home | Libraries | People | FAQ | More |
In general, it is a good practice to write any test as independent as possible from any other, there are however cases where a dependency cannot be avoided and an order for executing the tests is needed.
In the general setup and for any two test cases TA
and TB
, TB
should not take for granted that TA
has already executed, even if TA
is declared before TB
in
the same translation unit. The only ordering-related guarantee that Unit
Test Framework makes by default is that if test cases TA
and TB
are declared in the same test suite, no test case (call it TX
) from any other test suite is executed
between TA
and TB
, even if the declaration of TX
appears between the declarations of
TA
and TB
.
In other words, all tests from a suite are executed in one go, even if the
test suite namespace is opened multiple times.
Even though the order is not guaranteed, it may accidentally be preserved
across the different runs. In order to make sure the test cases do not depend
on one another, the test module may be called with an additional command-line
argument, random
, to shuffle the tests unit
ordering and to be more robust against an erroneous implicit ordering.
If there exist a dependency between test units, and an ordering is required between the execution of those tests, it has to be declared explicitly. Dependencies in the Unit Test Framework affect two dimensions of test units, which are:
Decorator
depends_on
associates the decorated
test case (call it TB
) with
another test case (call it TA
)
specified by name. This affects the processing the test tree in two ways:
TA
is
ordered to be run before TB
,
irrespective of the order in which they were declared or added to the
test tree,
TB
is skipped if TA
is either
disabled or skipped or is executed and marked as failed.
Code |
---|
#define BOOST_TEST_MODULE decorator_07 #include <boost/test/included/unit_test.hpp> namespace utf = boost::unit_test; // test1 and test2 defined at the bottom BOOST_AUTO_TEST_CASE(test3, * utf::depends_on("s1/test1")) { BOOST_TEST(false); } BOOST_AUTO_TEST_CASE(test4, * utf::depends_on("test3")) { BOOST_TEST(false); } BOOST_AUTO_TEST_CASE(test5, * utf::depends_on("s1/test2")) { BOOST_TEST(false); } BOOST_AUTO_TEST_SUITE(s1) BOOST_AUTO_TEST_CASE(test1) { BOOST_TEST(true); } BOOST_AUTO_TEST_CASE(test2, * utf::disabled()) { BOOST_TEST(false); } BOOST_AUTO_TEST_SUITE_END() |
Output |
---|
> decorator_07 --report_level=detailed --log_level=all Running 4 test cases... Entering test module "decorator_07" test.cpp:31: Entering test suite "s1" test.cpp:33: Entering test case "test1" test.cpp:35: info: check true has passed test.cpp:33: Leaving test case "test1"; testing time: 100us test.cpp:31: Leaving test suite "s1"; testing time: 129us test.cpp:16: Entering test case "test3" test.cpp:18: error: in "test3": check false has failed test.cpp:16: Leaving test case "test3"; testing time: 48us test.cpp:26: Test case "test5" is skipped because dependency test case "s1/test2" is disabled test.cpp:21: Test case "test4" is skipped because dependency test case "test3" has failed Leaving test module "decorator_07"; testing time: 263us Test module "decorator_07" has failed with: 1 test case out of 4 passed 1 test case out of 4 failed 2 test cases out of 4 skipped 1 assertion out of 2 passed 1 assertion out of 2 failed Test case "test3" has failed with: 1 assertion out of 1 failed Test case "test4" was skipped Test case "test5" was skipped Test suite "s1" has passed with: 1 test case out of 1 passed 1 assertion out of 1 passed Test case "s1/test1" has passed with: 1 assertion out of 1 passed |
In the above scenario:
test3
is run
(and fails) because s1/test1
has been run and succeeded,
test4
is skipped because
test3
has failed,
test5
is skipped because
s1/test2
is disabled.