Dovetail Test Cases

This is the list of candidate test cases which have been proposed by community members for addition to the Dovetail test suite. These test cases will be evaluated under the Dovetail test case requirements, proposed for inclusion as patches in Gerrit against Dovetail, and reviewed by the Dovetail maintainers. Once approved, they will become part of the Dovetail test suite.

VIM Operations

ID	Test Case	Test Case Group	Description	Status	Gerrit References
1	Add/Delete image	VIM operations	OpenStack Glance test using Tempest	draft proposed	links to be added
2	Query/list images	VIM operations	do(for openstack, there are 2 kinds of images related, 1st the nova, compute-images-create/delete/get/list, 2nd Glance, images-v2-remove/update/share/list/delete/get, MatthewLi )
3	Create/Delete network	VIM operations	OpenStack Network(Neutron) test using Tempest
4	Create/Delete subnet	VIM operations	do
5	Create/Delete port	VIM operations	do
6	Start an instance	VIM operations	Start a VM/container, verify that it comes up / OpenStack Nova Tempest test
7	Stop an instance	VIM operations	Stop a VM/container, verify that it goes down/ / OpenStack Nova Tempest test
8	Show server details	VIM operations
9	List servers	VIM operations
	Update server	VIM operations
	Create server	VIM operations
	Delete server	VIM operations
	Update metadata items	VIM operations
	Stop and start server	VIM operations
	Reboot server	VIM operations
	Confirm resized server	VIM operations
	Resize server	VIM operations
	Live-migrate server	VIM operations
	List hosts	VIM operations
	Create flavor	VIM operations
	List flavors with details	VIM operations
	Show flavor details	VIM operations
	Create extra specs for a flavor	VIM operations
	Show an extra spec for a flavor	VIM operations
	Attach and detach a volume by Nova	VIM operations
	Attach and detach a volume by Cinder	VIM operations
			...fill out CRUD operations for instances, network (+ storage?)

NFVI/Networking

Test Case	Test Case Group	Description
vPing	smoke(functest)	two vms setup in the same subnet and can ping each other (ssh, userdata, IPv4, IPv6(afraid not support now MatthewLi))
vRouter		Two VMs in two different subnets, connected by a Neutron virtual router, can ping each other
Security rules		Verify that a security rule prevents a type of traffic, remove rule, verify that traffic passes, add it back, test that traffic is stopped
External routing		Verify that a VM can access hosts external to the VIM
		...fill out CRUD operations for "network/subnet/router/port" operations

NFVI/Compute

Test Case	Test Case Group	Description
yardstick_tc010 measure memory read latency using lmbench	nfvi/compute	measure memory read latency, No SLA, some adapted work is needed ---> CAN Measure measure memory read latency Comment: There was an agreement not to include any performance related tests to the conformance test suite. Comment: These tests are not related to platform compliance
yardstick_tc012 Measure memory read and write bandwidth using lmbench	nfvi/compute	No SLA Comment: There was an agreement not to include any performance related tests to the conformance test suite. Comment: These tests are not related to platform compliance
yardstick_tc014 Measure Processing speed using unixbench	nfvi/compute	No SLA Comment: There was an agreement not to include any performance related tests to the conformance test suite. Comment: These tests are not related to platform compliance
yardstick_tc055 number of cores, number of threads, available memory size and cache size	nfvi/compute	No SLA COmment: This is not relevant to compliance and conformance

NFVI/Storage

ID

Test Case

Test Case Group

Description

Status

Gerrit References

yardstick_tc005
Measure Storage IOPS, throughput and latency using fio

nfvi/storage

No SLA

Comment: There was an agreement not to include any performance related tests to the conformance test suite.

Comment: These tests are not related to platform compliance

High Availability (HA)

ID	Test Case	Test Case Group	Description	Status	Gerrit References
	OPNFV_YARDSTICK_TC019_HA This test case will verify the high availability of the service provided by OpenStack (like nova-api, neutro-server) on control node.
	OPNFV_YARDSTICK_TC025_HA This test case will verify the high availability of controller node. When one of the controller node abnormally shutdown, the service provided by it should be OK
	OPNFV_YARDSTICK_TC045 This test case will verify the high availability of the network service provided by OpenStack (neutro-server) on control node
	OPNFV_YARDSTICK_TC046 This test case will verify the high availability of the user service provided by OpenStack (keystone) on control node
	OPNFV_YARDSTICK_TC047 This test case will verify the high availability of the image service provided by OpenStack (glance-api) on control node
	OPNFV_YARDSTICK_TC048 This test case will verify the high availability of the volume service provided by OpenStack (cinder-api) on control node
	OPNFV_YARDSTICK_TC049 This test case will verify the high availability of the storage service provided by OpenStack (swift-proxy) on control node
	OPNFV_YARDSTICK_TC050 This test case will verify the high availability of control node. When one of the controller failed to connect the network, which breaks down the Openstack services on this node. These Openstack service should able to be accessed by other controller nodes, and the services on failed controller node should be isolated
	OPNFV_YARDSTICK_TC051 This test case will verify the high availability of control node. When the CPU usage of a specified controller node is stressed to 100%, which breaks down the Openstack services on this node. These Openstack service should able to be accessed by other controller nodes, and the services on failed controller node should be isolated
	OPNFV_YARDSTICK_TC052 This test case will verify the high availability of control node. When the disk I/O of a specified disk is blocked, which breaks down the Openstack services on this node. Read and write services should still be accessed by other controller nodes, and the services on failed controller node should be isolated
	OPNFV_YARDSTICK_TC053 This test case will verify the high availability of the load balance service(current is HAProxy) that supports OpenStack on controller node. When the load balance service of a specified controller node is killed, whether other load balancers on other controller nodes will work, and whether the controller node will restart the load balancer are checked
	OPNFV_YARDSTICK_TC054 This test case will verify the high availability for virtual ip in the environment. When master node of virtual ip is abnormally shutdown, connection to virtual ip and the services binded to the virtual IP it should be OK

IPv6

Test Case	Test Case Group	Description
IPv6 IPAM for an instance	IPv6	Verify DHCPv6 (stateful & stateless), SLAAC
Security rules	IPv6	IPv6 security rules for Neutron
IPv6 routing	IPv6	using switch/router, IPv6 service VM etc.
OPNFV_YARDSTICK_TC027_IPv6 connectivity To do a basic verification that IPv6 connectivity is within acceptable boundaries when ipv6 packets travel between hosts located on same or different compute blades.

VPN

Test Case	Test Case Group	Description
VPN operations	VPN CRUD	Create VPN instance and check command return code Check if VPN instance is present Create IETF VM interface and check return code Verify IETF VM interface Create VPN interface for IETF interface Verify VPN interface Verify FIB entry after create Delete VM VPN interface Verify after deleting VM VPN interface Delete VPN instance Verify after deleting VPN instance Delete VM IETF interface Verify after deleting VM IETF interface Verify FIB entry after delete
VPN operations	BGP VPN CRUD	Create BGPVPN Create BGPVPN with malformatted route target (e.g. ASN:NN) should fail. Create BGPVPN with invalid route target (e.g. 65536:0) should fail. Getting the VPN list works without producing an error. Updating an existing BGPVPN works. Displaying parameters of an existing BGPVPN works. Deleting a BGPVPN works. Associating an existing BGPVPN with a Neutron network works Getting the associated Neutron network works. Deleting the network association works. Associating an existing BGPVPN with a Neutron Router works Getting the associated Neutron router works. Deleting the router association works
Subnet connectivity	VPN DC	VPN provides connectivity between subnets. different Neutron subnets cannot reach each other unless they are connected to the same Router.
Tenant separation	VPN DC	VPNs enable using the same IP address ranges in different VPNs, which is an important feature for tenant separation in the DC. This test verifies if the correct VM is reached under a given IP address when IP addresses are used multiple times in the same DC.
Data Center Gateway integration	VPN Multisite	Data Center Gateway integration, check if routes are correctly exchanged between SDN controller and datacenter gateway
Inter-DC communication	VPN Multisite	Inter-DC communication, Check if communication between two data centers connected through an MPLS backbone works.

Fault Management

Test Case	Test Case Group	Description
Relaunch instance ?	Fault management	When an instance fails, can the platform restart the instance automatically?
Scale-up ?	Fault management	Can a new instance of a given type be launched when a certain load is hit?
Subscribe to a fault event	Fault management	Listen for an instance fault , and verify that when the instance is killed we hear about it

Space shortcuts

Page tree