Since dovetail project differ from those testing projects, such as (FUNCTEST, YARDSTICK), it is necessary to list terms used in project and give a detailed description to avoid confusion.
Here is the table showing current terms in project.
If you feel any word is unclear , you can put it into this table, so we can review and discuss these terms constantly
Welcome recommendations and comments.
| term | reference | description | recommend | status | comments |
|---|---|---|---|---|---|
| certification | certification file name in directory cert/ | management unit, collection of test case. dovetail tool runs only one certification at a time. | Recommend not using the word certification, nor cert, in our tools or documents. (to use "compliance" instead of "certification"? MatthewLi) | ||
| scenario | input parameter for dovetail tool | actually the same meaning as certification, seems like a redundant term | Lets not use the term scenario. Can we adopt "test suite" instead? (agree, should think out name to replace this "scenario", since in OPNFV, "scenario" has special meaning MatthewLi) | ||
| test case | test case file in directory testcase/ | running unit, definition of a test case in dovetail, may contains multiple sub test cases | |||
| sub test case | test case definition files | represent test cases from upstream projects (FUNCTEST/YARDSTICK/...) | propose removing this term. "test activity" maybe. | ||
| scripts | test case definition files | dovetail leverages some upstream projects to run tests, such as (FUNCTEST/YARDSTICK/...), scripts includes a list of attributes concerning a certain upstream project, like 'type', 'testcase' and 'sub_testcase_list' | propose using "test source" | ||
| script test case | parameter name in conf/dovetail_config.yml | a sub set of test cases from upstream projects ( FUNCTEST/YARDSTICK/... ), one script test case may contain several sub test cases | how does this differ from "test suite", is this a "test area"? (I'm not sure the initial idea of this name, but in current implementation, it means test case from other projects; for example, in dovetail.ipv6.tc001.yml , script test case is 'tempest_smoke_serial' which is smoke tier test case in functest ) | ||
| System Under Test (SUT) | The target of the test cases. | The defined functional entity to be evaluated during the execution of the test suite described. The SUT should be consistently scoped for any given test suite. | |||
| INSTALLER_IP | config item in functest_config.yml | Any ip address that can access the OPENSTACK APIs is OK, could be the external ip address of OPENSTACK | "SUT_IP" may be a candidate (agree SUT_IP, MatthewLi) | ||
| INSTALLER_TYPE | config item in functest_config.yml | It's used to indicated that which installer is used to deploy the SUT, dovetail may get rid of it until testing projects do not depend on it any more | ("SUT_TYPE" maybe MatthewLi) | ||
| compliance_set | |||||
| proposed_tests | |||||
| test area |
compliance_set
| proposed_tests |