...
- Mandatory Test Areas - all mandatory test areas are present
- Test-Case Count within Test Area - total number of test-cases are present in each test area
- Test-Case Pass Percentage - all tests have passed (100% pass rate)
- Log File Verification - inspect the log file for each test area (osinterop, ha, vping)
- SUT Information Verification - TBA when Dovetail cvp.0.9.0 is availableInfo Verification - validate the hardware and software endpoint info is present
1. Mandatory Test Areas
Validate that all mandatory test areas are included in the overall test suite. The required mandatory test areas are:
- osinterop
- vping
- ha
Login to the CVP OVP portal at:
Click on the 'My Results' tab in top-level navigation bar.
The CVP OVP administrator will ask for review volunteers using the cvp@opnfv.org email alias. The incoming results for review will be identified by the administrator with particular 'Test ID' and 'Owner' values. The corresponding CVP OVP portal result will will have a status of 'review'.
...
The 'Test ID' hyperlink toggles the view to a top-level listing of the results displayed above. Validate that osinterop, vping and ha test areas are all present within the view.
2. Test-Case Count within Test Area
Validate the test-case count within each test area. For the Danube release, this must break down as follows:outlined in the table below.
| Mandatory Test Area | Test-Case Count |
|---|---|
| osinterop | 205 |
| ha | 8 |
| vping | 2 |
| Total | 215 |
...
In the diagram above (from section 1), these counts can be gleaned from the numbers to the right of the test-cases. The total number is given for the osinterop (dovetail.osinterop.tc001) test area at 205. The vping (dovetail.vping.tc001, tc00x) and ha (dovetail.vpingha.tc002tc00x) and ha test test-cases are broken down separately with a line for each test-case. Directly above the 'Test Result Overview' listing there's a summary labelled 'Test Run Results' shown below.
...
An example of a listing that should flag a negative review is shown above. This overall output The mandatory total contains only 214 test-cases, which is missing one of the ha test-cases.
3. Test-Case Pass Percentage
All mandatory test-cases must pass. This can be validated in multiple ways. The below diagram of the 'Test Run Results' is one method and shows that 100% of the mandatory test-cases have passed. This value must not be lower than 100%.
...
Another method to check that all mandatory test-cases have passed is shown in the diagram below. The pass/total is given as a fraction and highlighted here in yellow. For the osinterop test area, the result must display [205/205] and [1/1] for and for each of the test-cases under the vping and ha test areas [1/1] must be displayed.
4. Log File Verification
Three log files must be verified for content within each mandatory test area. The log files for each of the test areas is noted in the table below.
...
The final validation is for the vping test area log file (functest.log). The two entries displayed in the diagrams below must be present in this log file.
- vping_userdata
- vping_ssh
...
5. SUT Info Verification
SUT information must be present in the results to validate that all required endpoint services and at least two controllers were present during test execution. For the results shown below, click the 'info' hyperlink in the SUT column to navigate to the SUT information page.
In the 'Endpoints' listing shown below for the SUT VIM component, ensure that services are present for identify, compute, image, volume and network at a minimum by inspecting the 'Service Type' column.
Inspect the 'Hosts' listing found below the Endpoints secion of the SUT info page and ensure at least two hosts are present, as two controllers are required the for the mandatory HA test-cases.