Anuket Project

  • Created by Unknown User (christopherprice), last modified by Unknown User (dneary) on Dec 07, 2016

You are viewing an old version of this page. View the current version.

Compare with Current View Page History

« Previous Version 24 Next »

This page is intended as a working draft of how we will look to leverage and develop test cases for the dovetail project.

Dovetail Test Suite Purpose and Goals

The dovetail test suite is intended to provide a method for validating the interfaces and behaviors of an NFVi platform according to the expected capabilities exposed in an OPNFV NVFi instance, and to provide a baseline of functionality to enable VNF portability across different OPNFV NFVi instances. All dovetail tests will be available in open source and will be developed on readily available open source test frameworks.

Working with the ETSI NFV TST 001 reference: http://www.etsi.org/deliver/etsi_gs/NFV-TST/001_099/001/01.01.01_60/gs_NFV-TST001v010101p.pdf
The Dovetail project will focus on tests validating a scope for a System Under Test (SUT) associated with Chapter 4.9 - NFV Infrastructure + VIM Under Test, as adapted for OPNFV (see figure below).  The test suite will also contain pre-deployment validation tests to ensure that any platform under evaluation is first in a steady state of readiness. (It is open to debate whether we should have tests validating a steady state, or whether we should simply specify preconditions for the steady state we expect to find)

Test case requirements

The following requirements should be fulfilled for all tests added to the Dovetail test suite:

  • Test cases should favour implementation of a published standard interface for validation
    • Where no standard is available provide API support references
    • If a standard exists and is not followed, an exemption method needs to be established
  • Test cases be documented
    • Use case specification
    • Test preconditions
    • Basic test flow execution descriptor
    • Post conditions and pass fail criteria
    • Parameter border test cases descriptions
    • Fault/Error test case descriptions  (optional)
  • Test cases must pass on OPNFV reference deployments
    • Tests must not require a specific NFVi platform composition or installation tool
    • Tests must not require unmerged patches to the relevant upstream projects in order to pass when run against OPNFV reference deployments
  • The certification test suite must not require access to OPNFV infrastructure or resources in order to pass

Dovetail Test Suite Structure

A dovetail test suite should have the following overall components and structure:  (stolen, if simplified a little, from IEEE)

  • Test Plan
    • The test plan should describe how the test will proceed, who will do the testing and what will be tested.
  • Test Design Specification
    • The design spec outlines what needs to be tested including such information as applicable standards, requirements, and tooling.
  • Test Case Specification
    • The Test Case spec describes the purpose of a test, required inputs and expected results, including step-by-step procedures pass/fail criteria.
  • Test Procedure Specification
    • Describes how to run the test, preconditions and procedural steps to be followed.
  • Test Log
    • A log of the tests run, by whom, when, and the results of the tests.
  • Test Report
    • The test report describes the tests that were run, any failures and associated bugs, detailing deviations from expectation.
  • Test Summary Report
    • Summary of all results including a test result assessment (to be defined).

Dovetail Test Suite Naming Convention

Test case naming and structure for dovetail, external facing naming sequences for compliance and certification test cases.

Dovetail Test Result Compilation, Storage and Authentication

Test execution identification, results evaluation, storage, identification and security for dovetail compliance and certification test cases.

Phasing the Dovetail Development Effort

While not all tests will be possible to develop at once the following approach is proposed for the development of dovetail test suites in a structured manner.

Dovetail phase 1

Dovetail should initially set out to provide validation of interfaces and behaviors common to an OPNFV NVFi.  This can be seen as a set of test cases that evaluate if a NVFi implementation is able to achieve a steady operational state covering the common behaviors expected of an OPNFV NFVi.  In this case the dovetail tests will focus on a SUT definition of VNFi & VIM as described in 4.9 of the ETSI NFV TST 001 specification.

Dovetail phase 2

Dovetail should further establish a set of test suites that validate additional desired OPNFV VNFi behaviours.  This may include for instance, deployment specific capabilities for edge or remote installations.  It may include the validation of functionality that is not yet common to all OPNFV VNFi scenario's.

In phase 2 it may also be possible that dovetail provides such services as application test suites to validate the behavior of applications in preparation for deployment on an OPNFV VNFi.  This may result in the definition of new SUT scopes for dovetail as described in of the ETSI NFV TST 001 specification.

Dovetail phase 3

hongbo: the same as those defined in the phase 1 and phase 2

 

  • No labels