Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

...

Reviewers must follow the checklist below checklist to ensure review consistency for the OPNFV Danube release at a minimum.

  1. Mandatory Test Areas - all mandatory test areas are present
  2. Test-Case Count within Test Area - total number of test-cases are present in each test area
  3. Test-Case Pass Percentage - all tests have passed (100% pass rate)
  4. Log File Verification - inspect the log file for each test area (osinterop, ha, vping)
  5. SUT Information Verification - TBA when Dovetail cvp.0.9.0 is availableInfo Verification - validate the hardware and software endpoint info is present

1. Mandatory Test Areas

Validate that all mandatory test areas are included in the overall test suite. The required mandatory test areas are:

    • osinterop
    • vping
    • ha

Login to the CVP OVP portal at: 

https://cvpverified.opnfv.org

Click on the 'My Results' tab in top-level navigation bar.

The CVP OVP administrator will ask for review volunteers using the cvp@opnfv.org email alias. The incoming results for review will be identified by the administrator with a particular 'Test ID' , and 'Owner' and values. The corresponding OVP portal result will will have a status of 'review'.

...

The 'Test ID' hyperlink toggles the view to a top-level listing of the results displayed above. Validate that osinterop, vping and ha test areas are all present within the view. 

 

2. Test-Case Count within Test Area

Validate the test-case count within each test area. For the Danube release, this must break down as follows:outlined in the table below.

 

Mandatory Test AreaTest-Case Count
osinterop205
ha8
vping2
Total215

...

In the diagram above (from section 1), these counts can be gleaned from the numbers to the right of the test-cases. The total number is given for the osinterop (dovetail.osinterop.tc001) test area at 205. The vping (dovetail.vping.tc001, tc00x) and ha (dovetail.vpingha.tc002tc00x) and ha test test-cases are broken down separately with a line for each test-case. Directly above the 'Test Result Overview' listing there's a summary labelled 'Test Run Results' shown below.

...

An example of a listing that should flag a negative review is shown above. This overall output The mandatory total contains only 214 test-cases, which is missing one of the ha test-cases.

3. Test-Case Pass Percentage

All mandatory test-cases must pass. This can be validated in multiple ways. The below diagram of the 'Test Run Results' is one method and shows that 100% of the mandatory test-cases have passed. This value must not be lower than 100%.

...

Another method to check that all mandatory test-cases have passed is shown in the diagram below. The pass/total is given as a fraction and highlighted here in yellow. For the osinterop test area, the result must display [205/205] and [1/1] for and for each of the test-cases under the vping and ha test areas [1/1] must be displayed

4. Log File Verification

Three log files must be verified for content within each mandatory test area. The log files for each of the test areas is noted in the table below.

...

The final validation is for the vping test area log file (functest.log). The two entries displayed in the diagrams below must be present in this log file.

    • vping_userdata
    • vping_ssh

 

 

 

...

5. SUT Info Verification

SUT information must be present in the results to validate that all required endpoint services and at least two controllers were present during test execution. For the results shown below, click the 'info' hyperlink in the SUT column to navigate to the SUT information page.

Image Added

In the 'Endpoints' listing shown below for the SUT VIM component, ensure that services are present for identify, compute, image, volume and network at a minimum by inspecting the 'Service Type' column. 

Image Added

Inspect the 'Hosts' listing found below the Endpoints secion of the SUT info page and ensure at least two hosts are present, as two controllers are required the for the mandatory HA test-cases.