Dovetail Test Areas and Test Cases

This is the list of candidate test areas which have been proposed by community members for addition to the Dovetail test suite. These test areas will be evaluated under the Dovetail test case requirements, proposed for inclusion as patches in Gerrit against Dovetail, and reviewed by the Dovetail maintainers. Once approved, they will become part of the Dovetail test suite.

VIM Operations on compute

ID	Test Case	Test Case Group	Description	Status
1	Flavors CRUD using OpenStack Nova	VIM operations	OpenStack Nova test using Tempest	draft proposed
2	Start and stop server using OpenStack Nova	VIM operations	Start/stop a VM/container, verify that it comes up / OpenStack Nova Tempest test
4	Server CRUD using OpenStack Nova	VIM operations
5	Metadata items CRUD using OpenStack Nova	VIM operations
6	Reboot server	VIM operations
7	Confirm resized server	VIM operations
8	Resize server	VIM operations
9	Live-migrate server	VIM operations
10	List hosts	VIM operations
11	Extra specs CRUD for a flavor	VIM operations
12	Attach and detach a volume by Nova	VIM operations

VIM Operations on networking

ID	Test Case	Test Case Group	Status
1	Networks CRUD using OpenStack Neutron	VIM operations	draft proposed
2	Ports CRUD using OpenStack Neutron	VIM operations	draft proposed
3	Subnets CRUD using OpenStack Neutron	VIM operations	draft proposed
4	Floating IPs CRUD using OpenStack Neutron	VIM operations	draft proposed
5	Routers CRUD using OpenStack Neutron	VIM operations	draft proposed

VIM Operations on volumes

ID	Test Case	Test Case Group	Description	Status	Gerrit References
1	Persistence of files using volume			draft proposed
2	Attach and detach a volume by Cinder	VIM operations

VIM Operations on images

ID

Test Case

Test Case Group

Description

Status

Gerrit References

1

VNF image CRUD using Openstack Glance

VIM operations

OpenStack Glance test using Tempest

draft proposed

https://gerrit.opnfv.org/gerrit/#/c/22855/

https://gerrit.opnfv.org/gerrit/#/c/23843/

VIM Operations on identity

ID	Test Case	Test Case Group	Description	Status	Gerrit References
1	Role CRD using Openstack Keystone	VIM operations	OpenStack Keystone test using Tempest	draft proposed

NFVI/Networking

Test Case	Test Case Group	Description
vPing	smoke(functest)	two vms setup in the same subnet and can ping each other (ssh, userdata, IPv4, IPv6(afraid not support now MatthewLi))
vRouter		Two VMs in two different subnets, connected by a Neutron virtual router, can ping each other
Security rules		Verify that a security rule prevents a type of traffic, remove rule, verify that traffic passes, add it back, test that traffic is stopped
External routing		Verify that a VM can access hosts external to the VIM
		...fill out CRUD operations for "network/subnet/router/port" operations

NFVI/Compute

Test Case	Test Case Group	Description
yardstick_tc010 measure memory read latency using lmbench	nfvi/compute	measure memory read latency, No SLA, some adapted work is needed ---> CAN Measure measure memory read latency Comment: There was an agreement not to include any performance related tests to the conformance test suite. Comment: These tests are not related to platform compliance
yardstick_tc012 Measure memory read and write bandwidth using lmbench	nfvi/compute	No SLA Comment: There was an agreement not to include any performance related tests to the conformance test suite. Comment: These tests are not related to platform compliance
yardstick_tc014 Measure Processing speed using unixbench	nfvi/compute	No SLA Comment: There was an agreement not to include any performance related tests to the conformance test suite. Comment: These tests are not related to platform compliance
yardstick_tc055 number of cores, number of threads, available memory size and cache size	nfvi/compute	No SLA COmment: This is not relevant to compliance and conformance

NFVI/Storage

ID

Test Case

Test Case Group

Description

Status

Gerrit References

yardstick_tc005
Measure Storage IOPS, throughput and latency using fio

nfvi/storage

No SLA

Comment: There was an agreement not to include any performance related tests to the conformance test suite.

Comment: These tests are not related to platform compliance

High Availability (HA)

ID	Test Case	Test Case Group	Description	Status	Gerrit References
	OPNFV_YARDSTICK_TC019_HA This test case will verify the high availability of the service provided by OpenStack (like nova-api, neutro-server) on control node.
	OPNFV_YARDSTICK_TC025_HA This test case will verify the high availability of controller node. When one of the controller node abnormally shutdown, the service provided by it should be OK
	OPNFV_YARDSTICK_TC045 This test case will verify the high availability of the network service provided by OpenStack (neutro-server) on control node
	OPNFV_YARDSTICK_TC046 This test case will verify the high availability of the user service provided by OpenStack (keystone) on control node
	OPNFV_YARDSTICK_TC047 This test case will verify the high availability of the image service provided by OpenStack (glance-api) on control node
	OPNFV_YARDSTICK_TC048 This test case will verify the high availability of the volume service provided by OpenStack (cinder-api) on control node
	OPNFV_YARDSTICK_TC049 This test case will verify the high availability of the storage service provided by OpenStack (swift-proxy) on control node
	OPNFV_YARDSTICK_TC050 This test case will verify the high availability of control node. When one of the controller failed to connect the network, which breaks down the Openstack services on this node. These Openstack service should able to be accessed by other controller nodes, and the services on failed controller node should be isolated
	OPNFV_YARDSTICK_TC051 This test case will verify the high availability of control node. When the CPU usage of a specified controller node is stressed to 100%, which breaks down the Openstack services on this node. These Openstack service should able to be accessed by other controller nodes, and the services on failed controller node should be isolated
	OPNFV_YARDSTICK_TC052 This test case will verify the high availability of control node. When the disk I/O of a specified disk is blocked, which breaks down the Openstack services on this node. Read and write services should still be accessed by other controller nodes, and the services on failed controller node should be isolated
	OPNFV_YARDSTICK_TC053 This test case will verify the high availability of the load balance service(current is HAProxy) that supports OpenStack on controller node. When the load balance service of a specified controller node is killed, whether other load balancers on other controller nodes will work, and whether the controller node will restart the load balancer are checked
	OPNFV_YARDSTICK_TC054 This test case will verify the high availability for virtual ip in the environment. When master node of virtual ip is abnormally shutdown, connection to virtual ip and the services binded to the virtual IP it should be OK

IPv6

Test Case	Test Case Group	Description	Gerrit References
IPv6 IPAM for an instance	IPv6	Verify DHCPv6 (stateful & stateless), SLAAC	https://gerrit.opnfv.org/gerrit/#/c/20719/ https://gerrit.opnfv.org/gerrit/#/c/23843/
Security rules	IPv6	IPv6 security rules for Neutron
IPv6 routing	IPv6	using switch/router, IPv6 service VM etc.
OPNFV_YARDSTICK_TC027_IPv6 connectivity To do a basic verification that IPv6 connectivity is within acceptable boundaries when ipv6 packets travel between hosts located on same or different compute blades.

VPN

Test Case	Test Case Group	Description
VPN operations	VPN CRUD	Create VPN instance and check command return code Check if VPN instance is present Create IETF VM interface and check return code Verify IETF VM interface Create VPN interface for IETF interface Verify VPN interface Verify FIB entry after create Delete VM VPN interface Verify after deleting VM VPN interface Delete VPN instance Verify after deleting VPN instance Delete VM IETF interface Verify after deleting VM IETF interface Verify FIB entry after delete
VPN operations	BGP VPN CRUD	Create BGPVPN Create BGPVPN with malformatted route target (e.g. ASN:NN) should fail. Create BGPVPN with invalid route target (e.g. 65536:0) should fail. Getting the VPN list works without producing an error. Updating an existing BGPVPN works. Displaying parameters of an existing BGPVPN works. Deleting a BGPVPN works. Associating an existing BGPVPN with a Neutron network works Getting the associated Neutron network works. Deleting the network association works. Associating an existing BGPVPN with a Neutron Router works Getting the associated Neutron router works. Deleting the router association works
Subnet connectivity	VPN DC	VPN provides connectivity between subnets. different Neutron subnets cannot reach each other unless they are connected to the same Router.
Tenant separation	VPN DC	VPNs enable using the same IP address ranges in different VPNs, which is an important feature for tenant separation in the DC. This test verifies if the correct VM is reached under a given IP address when IP addresses are used multiple times in the same DC.
Data Center Gateway integration	VPN Multisite	Data Center Gateway integration, check if routes are correctly exchanged between SDN controller and datacenter gateway
Inter-DC communication	VPN Multisite	Inter-DC communication, Check if communication between two data centers connected through an MPLS backbone works.

Fault Management(Doctor)

Test Case	Test Case Group	Description
Relaunch instance ?	Fault management	When an instance fails, can the platform restart the instance automatically?
Scale-up ?	Fault management	Can a new instance of a given type be launched when a certain load is hit?
Subscribe to a fault event	Fault management	Listen for an instance fault , and verify that when the instance is killed we hear about it

KVM

ID	Test Case	Test Case Group	Description	Status	Gerrit References
1	evaluate the KVM virtualization capability

Parser

ID	Test Case	Test Case Group	Description	Status	Gerrit References
1	verify the function of Yang-to-Tosca in Parser

virtual Traffic Classifier

ID	Test Case	Test Case Group	Description	Status	Gerrit References
1	To verify that a newly instantiated vTC is ‘alive’ and functional and its instantiation is correctly supported by the infrastructure
2	To verify that a newly instantiated vTC is ‘alive’ and functional and its instantiation is correctly supported by the infrastructure in the presence of noisy neighbours.

Copper

ID	Test Case	Test Case Group	Description	Status	Gerrit References
1	policy management

Promise

ID	Test Case	Test Case Group	Description	Status	Gerrit References
1	Resource reservation

Multisite

ID	Test Case	Test Case Group	Description	Status	Gerrit References
1	multiple connected OpenStack deployments, of different sizes

SDN controller(ODL)

ID	Test Case	Test Case Group	Description	Status	Gerrit References
1

SDN controller(ONOS)

ID	Test Case	Test Case Group	Description	Status	Gerrit References
1

Space shortcuts

Page tree