Overview Illustration

System Under Test (SUT)

The SUT are products from vendors (not OPNFV release artifacts themselves). Development tooling, such as CI/CD, is not part of the SUT. The vendors can bring up the SUT to a pre-Dovetail-test state in any way they choose. Dovetail will provide documentation to help do so.

The SUT consists of NFVI software, VIM software, and necessary hardware in one System Under Test. The hardware should follow Pharos guidelines. Vendors can use their own hardware, or third party or white box hardware to be tested as a whole. And if the combined whole passes the test suite, the CVP label applies to the software.

In the Danube cycle, Dovetail does not plan to test hardware-only systems.

Test Suite

Dovetail Danube scope is a subset of functionalities contained in the OPNFV Danube release. The C&C Committee has further reduced the scope for Danube cycle to be NFVI+VIM on functions and interfaces OVP document. And for example, performance testing is currently out of scope in Danube.

All test cases in the Dovetail test suite must meet Dovetail Test Case Requirements.

Proposals for inclusion of test areas can be made by completing this worksheet Dovetail Test Area / Test Case Worksheet and the associated analysis work. These proposals will then be reviewed in Dovetail project weekly, following the scope OVP document and Dovetail Test Case Requirements.

The work of constructing the Dovetail test suite is tracked in JIRA ticket:  https://jira.opnfv.org/browse/DOVETAIL-345.

The resulting test suite will be defined by compliance_set.yml file, and maintained in wiki for reader's convenience: Dovetail Test Areas and Test Cases. (Warning: the current wiki page ls not yet review and inaccurate.)

Working with Upstream

Openstack Refstack/Defcore https://refstack.openstack.org/#/guidelines (version 2016.8) has been integrated into the Dovetail test suite. Collaborative working relationship is being developed with Openstack Interoperability Working Group on a continuous basis. In future, we may collaborate to develop NFV extensions to refstack, for instance.

Test cases in the refstack guidelines of the version corresponding to the Openstack verion integrated in OPNFV (for Danube it is Newton) are integrated into Dovetail as is (currently it is version 2016.8). While it is possible that, under enexpected circumstances, some test cases may be found unsuitable for execution in OPNFV environment and may have to be skipped, we do not see such cases today and expect them to be rare exceptions. Should such exceptions occur, they would need to be reviewed by Dovetail for particular reasons, documented clearly and agreed to.

We may consider working with other upstream communities in Euphrates in a similar fashion.

Long Term Road Map

Desired features for Dovetail, if not possible or not accepted in Danube, should be road-mapped using Jira enhancement tickets. Features targeted for E release should start immediately in order to influence E release planning that is ongoing in the community.

EUAG and other user inputs should be incorporated into the road map using the same Jira backlog/to-do ticket mechanism. Such request tickets should be written a JIRA story with sufficient detail for the reader/developers to understand clearly. 

Dovetail's E cycle planning will be based on these Jira backlog/to-do tickets. Please use fixVersion "Future Release" instead of "Dovetail Danube".

Deliverables and JIRA Tasks (fixVersion "Dovetail Danube"):

1. A stand-alone Dovetail client software, to be delivered to and used by testers/vendors as
   1. Dorker container
   2. Git clone python source
      Tracked by JIRA tickets: tbd
2. A User Guide for the client software. Tracked by JIRA ticket: DOVETAIL-73.
3. Backend DB and web server software, and web UI software, for administrators and for reviewers and others to access test results. This item is TBD pending on what falls under LF and what falls under Dovetail.
   1. All backend server software and instructions for someone to create and administer the server
   2. Web UI for test result viewers (tbd)
      Tracked by JIRA tickets: tbd
4. Test strategy document: http://ircbot.wl.linuxfoundation.org/meetings/opnfv-meeting/2017/opnfv-meeting.2017-03-17-13.00.log.html /  http://artifacts.opnfv.org/dovetail/review/30811/testing_user_teststrategy/index.html. Tracked by JIRA tickets: 

   Jira
   server OPNFV JIRA serverId 96acfcf2-db1a-3859-891e-03a53e9315b0 key DOVETAIL-352

5. Finalized Dovetail test suite: 
   1. test requirements wiki
   2. test area/test case list wiki
   3. compliance_set.yml; 
   4. Test suite Document.
      Tracked by JIRA tickets: DOVETAIL-345.
6. QA: testing Dovetail client and server for meeting feature and quality requirements. Tracked by JIRA tickets; 

   Jira
   server OPNFV JIRA serverId 96acfcf2-db1a-3859-891e-03a53e9315b0 key DOVETAIL-180

7. Bug fixes.
8. Beta testing
9. Release.

To be updated. It is not accurate nor complete at this time.

Release "Dovetail Danube": https://jira.opnfv.org/projects/DOVETAIL/versions/11001

Kanban board for the overall Dovetail Danube release: https://jira.opnfv.org/secure/RapidBoard.jspa?rapidView=149&selectedIssue=DOVETAIL-345&quickFilter=371

Kanban board for tasks related test suite (Testarea & Tesecase): https://jira.opnfv.org/secure/RapidBoard.jspa?rapidView=147&quickFilter=357