Dovetail Test Areas and Test Cases

This is the list of candidate test areas which have been proposed by community members for addition to the Dovetail test suite. These test areas will be evaluated under the Dovetail test case requirements, proposed for inclusion as patches in Gerrit against Dovetail, and reviewed by the Dovetail maintainers. Once approved, they will become part of the Dovetail test suite.

VIM Operations on compute

ID	Type	Test Case	Description	Pre-condition & requirements	Status	Gerrit References
1	VIM operations on Compute	Create and show root certificates	Create and show root certificates	VIM: Openstack Components: Keystone, Nova
2	VIM operations on Compute	Flavors CRD ops	Create, delete, list and show flavors	VIM: Openstack Components: Keystone, Nova
3	VIM operations on Compute	Floating IPs CRUD ops	Allocate, delete, associate, disassociate, show and list floating ips	VIM: Openstack Components: Keystone, Nova, Neutron
4	VIM operations on Compute	Image metadata CRUD ops	List, set, update, show, delete image metadatas	VIM: Openstack Components: Keystone, Nova
5	VIM operations on Compute	Images CRUD ops	Delete, list, show and create images	VIM: Openstack Components: Keystone, Nova, Glance
6	VIM operations on Compute	Keypairs CRUD ops	Create, list, delete and show keypairs	VIM: Openstack Components: Keystone, Nova
7	VIM operations on Compute	Show absolute limits	Show absolute limits	VIM: Openstack Components: Keystone, Nova
8	VIM operations on Compute	Security group rules CRUD ops	Create, delete and list security group rules	VIM: Openstack Components: Keystone, Nova
9	VIM operations on Compute	Security group CRUD ops	Create, delete, show, update and list security groups	VIM: Openstack Components: Keystone, Nova
10	VIM operations on Compute	Interfaces CRUD ops	Create, delete, show and list interfaces; attach and deattach ports to servers	VIM: Openstack Components: Keystone, Nova, Neutron
11	VIM operations on Compute	List availability zones	List of availability zones and available services	VIM: Openstack Components: Keystone, Nova
12	VIM operations on Compute	Servers CRD ops	Create, delete, list and get servers	VIM: Openstack Components: Keystone, Nova, Neutron
13	VIM operations on Compute	Device tagging	Verifty device tagging metadata of servers	VIM: Openstack Components: Keystone, Nova, Neutron, Glance, Neutron
14	VIM operations on Compute	Rebuild, resize and update server	Rebuild, resize and update server	VIM: Openstack Components: Keystone, Nova
15	VIM operations on Compute	List and show instance actions	List and show instance actions	VIM: Openstack Components: Keystone, Nova
16	VIM operations on Compute	Multiple create servers	Verify whether reservation id is in the response body of creating servers	VIM: Openstack Components: Keystone, Nova
17	VIM operations on Compute	Server actions	Change server passwords; reboot, rebuild, resize, pause, unpause, suspend, resume, shelve, unshelve, stop, start, lock, unlock server; create backup; get console oupt; show vnc console;	VIM: Openstack Components: Keystone, Nova, Glance
18	VIM operations on Compute	List server addresses	List server addresses	VIM: Openstack Components: Keystone, Nova, Neutron
19	VIM operations on Compute	Server group	Create/delete server-groups with different policies. policies = affinity/anti-affinity It also adds the tests for list and get details of server-groups	VIM: Openstack Components: Keystone, Nova
20	VIM operations on Compute	Server metadata CRUD ops	List, set, update, show, delete server metadata	VIM: Openstack Components: Keystone, Nova
21	VIM operations on Compute	Delete and get server password	Delete and get server password	VIM: Openstack Components: Keystone, Nova
22	VIM operations on Compute	Server personality ops	Create and rebuild servers with personality	VIM: Openstack Components: Keystone, Nova
23	VIM operations on Compute	Rescue and unrescue servers	Rescue and unrescue servers	VIM: Openstack Components: Keystone, Nova
24	VIM operations on Compute	Create server with special metadata	Create server with admin password; create server with existing server name; create specify keypair; update server name; create server with IPv6 only	VIM: Openstack Components: Keystone, Nova
25	VIM operations on Compute	List virtual interfaces	List virtual interfaces	VIM: Openstack Components: Keystone, Nova, Neutron
26	VIM operations on Compute	Attach and deattach volume	Attach and deattach volume; list and show volume attachments; delete attached volume	VIM: Openstack Components: Keystone, Nova, Cinder
27	VIM operations on Compute	Volume snapshot CRD operations	Create, show, list and delete volume snapshots	VIM: Openstack Components: Keystone, Nova, Cinder
28	VIM operations on Compute	Volumes CRUD ops	Create, show, list, update and delete volumes	VIM: Openstack Components: Keystone, Nova, Cinder
29	VIM operations on Compute	Show and list extensions	Show and list extensions	VIM: Openstack Components: Keystone, Nova
30	VIM operations on Compute	Live block migration	Migrating to an invalid host should not change the status	VIM: Openstack Components: Keystone, Nova
31	VIM operations on Compute	List networks	List networks	VIM: Openstack Components: Keystone, Nova, Neutron
32	VIM operations on Compute	Quotas CRUD ops	Show, create, update and delete quotas	VIM: Openstack Components: Keystone, Nova
33	VIM operations on Compute	List and show tenant networks	List and show tenant networks	VIM: Openstack Components: Keystone, Nova, Neutron
34	VIM operations on Compute	List and show API versions	List and show API versions	VIM: Openstack Components: Keystone, Nova
35	VIM operations on Compute	Agents CRUD	Create, update, delete and list agents	VIM: Openstack Components: Keystone, Nova
36	VIM operations on Compute	Aggregate CRUD	Create, update, delete, show and list agents	VIM: Openstack Components: Keystone, Nova
37	VIM operations on Compute	Auto-allocating networks	Tests auto-allocating networks with the v2.37 microversion. The tenant must not have any network resources available to it so we can make sure that Nova calls to Neutron to automatically allocate the network topology.	VIM: Openstack Components: Keystone, Nova, Neutron
38	VIM operations on Compute	Fixed ips	List, reserve and unreserve fixed ips	VIM: Openstack Components: Keystone, Nova
39	VIM operations on Compute	Flavors access	Add and remove Flavor Access require admin privileges	VIM: Openstack Components: Keystone, Nova
40	VIM operations on Compute	Flavor Extra Spec API extension	SET, UNSET, UPDATE Flavor Extra specs require admin privileges. GET Flavor Extra specs can be performed even by without admin privileges.	VIM: Openstack Components: Keystone, Nova
41	VIM operations on Compute	Floating IPs Bulk CRUD	Create, update, delete, and list Floating IPs Bulk that require admin privileges	VIM: Openstack Components: Keystone, Nova, Neutron
42	VIM operations on Compute	Host CRUD negative	Create, update, delete, show and list hosts using admin privileges	VIM: Openstack Components: Keystone, Nova
43	VIM operations on Compute	List and show hosts	List and show hosts using admin privileges	VIM: Openstack Components: Keystone, Nova
44	VIM operations on Compute	List and show hypervisors	List and show hypervisors using admin privileges	VIM: Openstack Components: Keystone, Nova
45	VIM operations on Compute	Instance usage audit log negative	Show instance usage audit log	VIM: Openstack Components: Keystone, Nova
46	VIM operations on Compute	Instance usage audit log	Show and list instance usage audit log	VIM: Openstack Components: Keystone, Nova
47	VIM operations on Compute	Live block migration	Live migration between two hosts	VIM: Openstack Components: Keystone, Nova
48	VIM operations on Compute	Migrations admin	Live migrations and cold migrations	VIM: Openstack Components: Keystone, Nova
49	VIM operations on Compute	Nova network API	List and show network	VIM: Openstack Components: Keystone, Nova
50	VIM operations on Compute	Servers create ops on multinodes	Create servers on same host; create servers on different hosts	VIM: Openstack Components: Keystone, Nova
51	VIM operations on Compute	List and show services	List and show services	VIM: Openstack Components: Keystone, Nova
52	VIM operations on Compute	List and show tenant usage	List and show usage for tenants	VIM: Openstack Components: Keystone, Nova
53	VIM operations on Compute	Swap volumes	Swapp volumes with admin user	VIM: Openstack Components: Keystone, Nova, Cinder

VIM Operations on networking

ID	Type	Test Case	Description	Pre-condition & requirements	Status	Gerrit References
1	VIM operations on Network	Ports CRU ops with allowed address pair	Create, list, update and show ports with allowed address pair	VIM: Openstack Components: Keystone, Neutron
2	VIM operations on Network	DHCP + IPv6 service	Test DHCPv6 specific features using SLAAC, stateless and stateful settings for subnets. Also it shall check dual-stack functionality (IPv4 + IPv6 together). The tests include: generating of SLAAC EUI-64 address in subnets with various settings receiving SLAAC addresses in combinations of various subnets receiving stateful IPv6 addresses addressing in subnets with router	VIM: Openstack Components: Keystone, Neutron
3	VIM operations on Network	List all available extensions	List available extensions for the project	VIM: Openstack Components: Keystone, Neutron
4	VIM operations on Network	Ports CRU ops with extra DHCP options	Create, list, show and update ports with the Extra DHCP options	VIM: Openstack Components: Keystone, Neutron
5	VIM operations on Network	Floating IPs CRUD ops	Create, update, delete, list, show and associate Floating IPs	VIM: Openstack Components: Keystone, Neutron
6	VIM operations on Network	Metering extensions CRD ops	List, Show, Create, Delete Metering labels List, Show, Create, Delete Metering labels rules	VIM: Openstack Components: Keystone, Neutron
7	VIM operations on Network	Networks and subnets CRUD ops	create a network for a project list project's networks show a project network details create a subnet for a project list project's subnets show a project subnet details network update subnet update delete a network also deletes its subnets list external networks All subnet tests are run once with ipv4 and once with ipv6.;	VIM: Openstack Components: Keystone, Neutron
8	VIM operations on Network	Ports CRUD ops	Create, delete, list, show and update ports	VIM: Openstack Components: Keystone, Neutron
9	VIM operations on Network	Routers CRUD ops	Create, delete, list, show and update routers; add and remove router interface	VIM: Openstack Components: Keystone, Neutron
10	VIM operations on Network	Security groups CRUD ops	Create, delete, list, show and update security groups;Create, delete, show security group rules;	VIM: Openstack Components: Keystone, Neutron
11	VIM operations on Network	List service providers	List service providers	VIM: Openstack Components: Keystone, Neutron
12	VIM operations on Network	Subnet pools CRUD ops	Create, delete, list, show and update subnet pools	VIM: Openstack Components: Keystone, Neutron
13	VIM operations on Network	List API versions	List API versions	VIM: Openstack Components: Keystone, Neutron
14	VIM operations on Network	Agents management	List, show and update agents	VIM: Openstack Components: Keystone, Neutron
15	VIM operations on Network	DHCP agent scheduler	List DHCP agents on hosting network; List networks hosted by one DHCP; Add and remove networks from DHCP agents	VIM: Openstack Components: Keystone, Neutron
16	VIM operations on Network	External network CRUD ops	Create, delete, list, show and update external networks	VIM: Openstack Components: Keystone, Neutron
17	VIM operations on Network	L3 agent scheduler	List routers that the given L3 agent is hosting. List L3 agents hosting the given router. Add and Remove Router to L3 agent	VIM: Openstack Components: Keystone, Neutron
18	VIM operations on Network	Quotas CRU ops	List, show, update, set, reset quotas	VIM: Openstack Components: Keystone, Neutron
19	VIM operations on Network	Router DVR	Create Distributed Virtual Routing (DVR) router; create a centralized router; update a centralized router to DVR router	VIM: Openstack Components: Keystone, Neutron

VIM Operations on volumes

ID	Type	Test Case	Description	Pre-condition & requirements	Status	Gerrit References
1	VIM operations on Volume	List availability zones	List availability zones	VIM: Openstack Components: Keystone, Nova, Cinder
2	VIM operations on Volume	List extensions	List of all extensions	VIM: Openstack Components: Keystone, Nova, Cinder
3	VIM operations on Volume	Snapshot metadata CRUD ops	Create, show, update, delete metadata; Create, show, update, delete metadata items;	VIM: Openstack Components: Keystone, Nova, Cinder
4	VIM operations on Volume	Show volume absolute limits	Show volume absolute limits for a tenant	VIM: Openstack Components: Keystone, Cinder
5	VIM operations on Volume	Volume metadata CRUD ops	Create, show, update, delete volume metadata	VIM: Openstack Components: Keystone, Cinder
6	VIM operations on Volume	Volume transfer CRUD ops	Create, show, update, delete and list volume transfers	VIM: Openstack Components: Keystone, Cinder
7	VIM operations on Volume	Volume actions	Attach and deattach volume to an instance; Verify that a volume bootable flag is retrieved; Verify that a volume's attachment information is retrieved; upload volume; reserve and unreserve volumes; update volume; Reset volume status; force delete volumes	VIM: Openstack Components: Keystone, Cinder, Glance
8	VIM operations on Volume	Volume backup CRUD ops	Create, show, list, restore, delete volume backups;export and import volume backups; reset backup status	VIM: Openstack Components: Keystone, Cinder
9	VIM operations on Volume	Volumes clone	Creates a volume from another volume passing a size different from the source volume; Create a volume from the bootable volume	VIM: Openstack Components: Keystone, Cinder
10	VIM operations on Volume	Extend volume	Extend volume	VIM: Openstack Components: Keystone, Cinder
11	VIM operations on Volume	Volumes CRUD ops	Create, show, update, list and delete volumes	VIM: Openstack Components: Keystone, Cinder, Glance
12	VIM operations on Volume	Volumes snapshot CRUD ops	Create a snapshot when volume status is in-use; Create, show, list, update and delete snapshots;	VIM: Openstack Components: Keystone, Cinder
13	VIM operations on Volume	Image metadata RUD ops	Update and show, delete image metadata	VIM: Openstack Components: Keystone, Nova, Cinder, Glance
14	VIM operations on Volume	Create and show volume with Mutil backend	Show volume id which created by type without prefix;Show volume id which created by type wit prefix	VIM: Openstack Components: Keystone, Cinder
15	VIM operations on Volume	Qos specs CRUD ops	Create, list, delete, show, associate, disassociate, set/unset qos-specs	VIM: Openstack Components: Keystone, Cinder
16	VIM operations on Volume	Snapshot actions	Reset and update snapshot status; force delete snapshot;	VIM: Openstack Components: Keystone, Cinder
17	VIM operations on Volume	List hosts	List hosts	VIM: Openstack Components: Keystone, Nova, Cinder
18	VIM operations on Volume	Volume quotas RUD ops	List, show,update and delete volume quotas	VIM: Openstack Components: Keystone, Cinder
19	VIM operations on Volume	List volume services	Volume service list requires admin privileges	VIM: Openstack Components: Keystone, Cinder
20	VIM operations on Volume	Create and update volume snapshot quotas	Create and update volume snapshot quotas	VIM: Openstack Components: Keystone, Cinder
21	VIM operations on Volume	Volume types extra specs CRUD ops	Create, show, delete, list and update volume types extra specs	VIM: Openstack Components: Keystone, Cinder
22	VIM operations on Volume	Volume types CRUD ops	Create, update, show, delete and list volume types	VIM: Openstack Components: Keystone, Cinder
23	VIM operations on Volume	User messages RD ops	List, show and delete user messsages	VIM: Openstack Components: Keystone, Cinder
24	VIM operations on Volume	List and show backends capabilities	List and show backends capabilities	VIM: Openstack Components: Keystone, Cinder
25	VIM operations on Volume	List and show volume pools	List and show volume pools	VIM: Openstack Components: Keystone, Cinder
26	VIM operations on Volume	Add and list volume type access	Add and list volume type access	VIM: Openstack Components: Keystone, Cinder

VIM Operations on images

ID	Type	Test Case	Description	Pre-condition & requirements	Status	Gerrit References
1	VIM operations on Image	Images member CRUD ops	Create, show, delete, update, list images member	VIM: Openstack Components: Keystone, Glance
2	VIM operations on Image	Metadata definition namespace property basic functionality	Create, show, update, delete and list metadata definition namespace property	VIM: Openstack Components: Keystone, Glance
3	VIM operations on Image	Metadata definition Namespaces basic functionality	Create, show, update, delete and list metadata definition namespace	VIM: Openstack Components: Keystone, Glance
4	VIM operations on Image	Metadata definition resource types basic functionality	Create, show, update, delete and list metadata definition resource types	VIM: Openstack Components: Keystone, Glance
5	VIM operations on Image	Show metadata definition schema	Show metadata definition schema	VIM: Openstack Components: Keystone, Glance
6	VIM operations on Image	Images CRUD ops	Register, show , delete, update and list images; upload and show the image file	VIM: Openstack Components: Keystone, Glance		https://gerrit.opnfv.org/gerrit/#/c/22855/ https://gerrit.opnfv.org/gerrit/#/c/23843/
7	VIM operations on Image	Create and delete image tags	Create and delete image tags	VIM: Openstack Components: Keystone, Glance

VIM Operations on identity

ID	Type	Test Case	Description	Pre-condition & requirements	Status	Gerrit References
1	VIM operations on Identity	List extensions	List all the extensions	VIM: Openstack Components: Keystone
2	VIM operations on Identity	API discovery features	Show API version; show API media types; show API status	VIM: Openstack Components: Keystone
3	VIM operations on Identity	Projects CRUD ops	List, create, delete and update projects; associate a user to a project	VIM: Openstack Components: Keystone
4	VIM operations on Identity	Token CRUD ops	Create, show, delete, rescope token; Note: An unscoped token can be requested, that token can be used to request a scoped token. The scoped token can be revoked, and the original token used to get a token in a different project.	VIM: Openstack Components: Keystone
5	VIM operations on Identity	Update password	User update own password	VIM: Openstack Components: Keystone
6	VIM operations on Identity	User ec2 credentials CRD ops	Create, show, list and delete user ec2 credentials	VIM: Openstack Components: Keystone
7	VIM operations on Identity	Credentials CRUD ops	Create, show, update, delete and list credentials	VIM: Openstack Components: Keystone
8	VIM operations on Identity	Default project id	1. create a domain 2. create a project in the domain 3. create a user in the domain, with the previous project as his default project 4. get roles and find the admin role 5. grant the admin role to the user on his project 6. create a new client with user's credentials 7. verify the user's token and see that it is scoped to the project	VIM: Openstack Components: Keystone
9	VIM operations on Identity	Domains CRUD ops	Create, update, delete and list domains	VIM: Openstack Components: Keystone
10	VIM operations on Identity	Endpoint CRUD ops	Create, update, delete, show and list endpoints	VIM: Openstack Components: Keystone
11	VIM operations on Identity	Groups CRUD ops	Create, update, delete, show and list groups; add, list and delete users in groups;	VIM: Openstack Components: Keystone
12	VIM operations on Identity	Inherits CRUD ops	Create, assign, list, check, revoke roles on domains user/domains group/projects user/projects group/domains/ project trees	VIM: Openstack Components: Keystone
13	VIM operations on Identity	Policy CRUD ops	List, create, update and delete policies	VIM: Openstack Components: Keystone
14	VIM operations on Identity	Regions CRUD ops	Create, show, update, delete and list regions	VIM: Openstack Components: Keystone
15	VIM operations on Identity	Roles CRUD ops	Create, show, update, delete and list roles; grant ,list and revoke role to user/group on project/domain/	VIM: Openstack Components: Keystone
16	VIM operations on Identity	Services CRUD ops	Create, show, update, delete and list services	VIM: Openstack Components: Keystone
17	VIM operations on Identity	Trusts CRU ops	Create, show, delete and list trusts; updates are not supported for trusts	VIM: Openstack Components: Keystone
18	VIM operations on Identity	Users CRUD ops	Create, update, show, delete and list users; Request for token authentication with/without a valid token in header	VIM: Openstack Components: Keystone

NFVI/compute

ID	Type	Test Case	Description	Pre-condition & requirements	Status	Gerrit References
1	NFVI	Basic minimum scenario test	1. Create image 2. Create keypair 3. Boot instance with keypair and get list of instances 4. Create volume and show list of volumes 5. Attach volume to instance and getlist of volumes 6. Add IP to instance 7. Create and add security group to instance 8. Check SSH connection to instance 9. Reboot instance 10. Check SSH connection to instance after reboot	VIM: Openstack Components: Keystone, Nova, Neutron, Glance, Cinder
2	Compute	OPNFV_YARDSTICK_TC010	Measure the memory read latency for varying memory sizes and strides. Whole memory hierarchy is measured including all levels of cache.	Colorado Scenario:due to floating IP,not support odl_l2, bgnvpn VIM: Openstack Components: Keystone, Nova, Neutron, Glance, Cinder, Heat
3	Compute	OPNFV_YARDSTICK_TC012	Measure the rate at which data can be read from and written to the memory (this includes all levels of memory).	Colorado Scenario: due to floating IP,not support odl_l2, bgnvpn VIM: Openstack Components: Keystone, Nova, Neutron, Glance, Cinder, Heat
4	Compute	OPNFV_YARDSTICK_TC014	To evaluate the IaaS processing speed with regards to score of single cpu running and parallel running The purpose is also to be able to spot trends. Test results, graphs and similar shall be stored for comparison reasons and product evolution understanding between different OPNFV versions and/or configurations.	Colorado Scenario: due to floating IP,not support odl_l2, bgnvpn VIM: Openstack Components: Keystone, Nova, Neutron, Glance, Cinder, Heat
5	Compute	OPNFV_YARDSTICK_TC055	To evaluate the IaaS compute capacity with regards to hardware specification, including number of cpus, number of cores, number of threads, available memory size and total cache size. Test results, graphs and similar shall be stored for comparison reasons and product evolution understanding between different OPNFV versions and/or configurations.	Colorado Scenario:due to floating IP,not support odl_l2, bgnvpn VIM: Openstack Components: Keystone, Nova, Neutron, Glance, Cinder, Heat
6	Compute	Aggregates basic ops	Creates an aggregate within an availability zone 1. Adds a host to the aggregate 2. Checks aggregate details 3. Updates aggregate's name 4. Removes host from aggregate 5. Deletes aggregate	VIM: Openstack Components: Keystone, Nova, Neutron, Glance, Cinder
7	Compute	Server advanced operations	This test case stresses some advanced server instance operations: * Resizing a volume-backed instance * Sequence suspend resume	VIM: Openstack Components: Keystone, Nova, Neutron, Glance, Cinder
8	Compute	Server basic operations	This smoke test case follows this basic set of operations: * Create a keypair for use in launching an instance * Create a security group to control network access in instance * Add simple permissive rules to the security group * Launch an instance * Perform ssh to instance * Verify metadata service * Verify metadata on config_drive * Terminate the instance	VIM: Openstack Components: Keystone, Nova, Neutron, Glance, Cinder
9	Compute	Server multinode	This is a set of tests specific to multinode testing	VIM: Openstack Components: Keystone, Nova, Neutron, Glance, Cinder
10	Compute	Shelve instance	The following is the scenario outline: * boot an instance and create a timestamp file in it * shelve the instance * unshelve the instance * check the existence of the timestamp file in the unshelved instance	VIM: Openstack Components: Keystone, Nova, Neutron, Glance, Cinder
11	Compute	Snapshot pattern	The following is the outline: * boot an instance and create a timestamp file in it * snapshot the instance * boot a second instance from the snapshot * check the existence of the timestamp file in the second instance	VIM: Openstack Components: Keystone, Nova, Neutron, Glance, Cinder
12	Compute	Stamp pattern	This test is for snapshotting an instance/volume and attaching the volume created from snapshot to the instance booted from snapshot. The following is the scenario outline: 1. Boot an instance "instance1" 2. Create a volume "volume1" 3. Attach volume1 to instance1 4. Create a filesystem on volume1 5. Mount volume1 6. Create a file which timestamp is written in volume1 7. Unmount volume1 8. Detach volume1 from instance1 9. Get a snapshot "snapshot_from_volume" of volume1 10. Get a snapshot "snapshot_from_instance" of instance1 11. Boot an instance "instance2" from snapshot_from_instance 12. Create a volume "volume2" from snapshot_from_volume 13. Attach volume2 to instance2 14. Check the existence of a file which created at 6. in volume2	VIM: Openstack Components: Keystone, Nova, Neutron, Glance, Cinder
13	Compute	Volume boot pattern	This test case attempts to reproduce the following steps: * Create in Cinder some bootable volume importing a image * Boot an instance from the bootable volume * Write content to the volume * Delete an instance and Boot a new instance from the volume * Check written content in the instance * Create a volume snapshot while the instance is running * Boot an additional instance from the new snapshot based volume * Check written content in the instance booted from snapshot	VIM: Openstack Components: Keystone, Nova, Neutron, Glance, Cinder

NFVI on network

ID	Type	Test Case	Description	Pre-condition & requirements	Status	Gerrit References
1	Network	vPing	two vms setup in the same subnet and can ping each other (ssh, userdata, IPv4, IPv6(afraid not support now MatthewLi))	VIM: Openstack Components: Keystone, Nova, Neutron, Glance
2	Network	vRouter	Two VMs in two different subnets, connected by a Neutron virtual router, can ping each other	VIM: Openstack Components: Keystone, Nova, Neutron, Glance
3	Network	Security rules	Verify that a security rule prevents a type of traffic, remove rule, verify that traffic passes, add it back, test that traffic is stopped	VIM: Openstack Components: Keystone, Nova, Neutron, Glance
4	Network	External routing	Verify that a VM can access hosts external to the VIM	VIM: Openstack Components: Keystone, Nova, Neutron, Glance
5	Network		...fill out CRUD operations for "network/subnet/router/port" operations	VIM: Openstack Components: Keystone, Nova, Neutron, Glance
6	Network	Network advanced server ops	Check VM connectivity after some advanced instance operations executed: * Stop/Start an instance * Reboot an instance * Rebuild an instance * Pause/Unpause an instance * Suspend/Resume an instance * Resize an instance	VIM: Openstack Components: Keystone, Nova, Neutron, Glance, Cinder
7	Network	Network basic operations	This smoke test suite assumes that Nova has been configured to boot VM's with Neutron-managed networking, and attempts to verify network connectivity as follows: There are presumed to be two types of networks: tenant and public. A tenant network may or may not be reachable from the Tempest host. A public network is assumed to be reachable from the Tempest host, and it should be possible to associate a public ('floating') IP address with a tenant ('fixed') IP address to facilitate external connectivity to a potentially unroutable tenant IP address.	VIM: Openstack Components: Keystone, Nova, Neutron, Glance, Cinder
8	Network	Network v6	1. Create network with subnets: 1.1. one IPv4 and 1.2. one or more IPv6 in a given address mode 2. Boot 2 VMs on this network 3. Allocate and assign 2 FIP4 4. Check that vNICs of all VMs gets all addresses actually assigned 5. Each VM will ping the other's v4 private address 6. If ping6 available in VM, each VM will ping all of the other's v6 addresses as well as the router's	VIM: Openstack Components: Keystone, Nova, Neutron, Glance, Cinder
9	Network	Security groups basic ops	This test case assumes that Nova has been configured to boot VM's with Neutron-managed networking, and attempts to verify cross tenant connectivity as follows ssh: in order to overcome "ip namespace", each tenant has an "access point" VM with floating-ip open to incoming ssh connection allowing network commands (ping/ssh) to be executed from within the tenant-network-namespace Tempest host performs key-based authentication to the ssh server via floating IP address connectivity test is done by pinging destination server via source server ssh connection. success - ping returns failure - ping_timeout reached multi-node: Multi-Node mode is enabled when CONF.compute.min_compute_nodes > 1. Tests connectivity between servers on different compute nodes. When enabled, test will boot each new server to different compute nodes.	VIM: Openstack Components: Keystone, Nova, Neutron, Glance, Cinder

NFVI on storage

ID	Type	Test Case	Description	Pre-condition & requirements	Status	Gerrit References
1	Storage	yardstick_tc005	Fio test is invoked in a host VM on a compute blade, a job file as well as parameters are passed to fio and fio will start doing what the job file tells it to do.	Colorado Scenario: due to floating IP,not support odl_l2, bgnvpn VIM: Openstack Components: Keystone, Nova, Neutron, Glance, Cinder, Heat
2	Storage	Encrypted Cinder volumes	This test is for verifying the functionality of encrypted cinder volumes. For both LUKS and cryptsetup encryption types, this test performs the following: * Creates an image * Boots an instance from the image * Creates an encryption type (as admin) * Creates a volume of that encryption type (as a regular user) * Attaches and detaches the encrypted volume to the instance	VIM: Openstack Components: Keystone, Nova, Neutron, Glance, Cinder

High Availability (HA)

ID	Test Case	Test Case Group	Description	Status	Gerrit References
	OPNFV_YARDSTICK_TC019_HA This test case will verify the high availability of the service provided by OpenStack (like nova-api, neutro-server) on control node.
	OPNFV_YARDSTICK_TC025_HA This test case will verify the high availability of controller node. When one of the controller node abnormally shutdown, the service provided by it should be OK
	OPNFV_YARDSTICK_TC045 This test case will verify the high availability of the network service provided by OpenStack (neutro-server) on control node
	OPNFV_YARDSTICK_TC046 This test case will verify the high availability of the user service provided by OpenStack (keystone) on control node
	OPNFV_YARDSTICK_TC047 This test case will verify the high availability of the image service provided by OpenStack (glance-api) on control node
	OPNFV_YARDSTICK_TC048 This test case will verify the high availability of the volume service provided by OpenStack (cinder-api) on control node
	OPNFV_YARDSTICK_TC049 This test case will verify the high availability of the storage service provided by OpenStack (swift-proxy) on control node
	OPNFV_YARDSTICK_TC050 This test case will verify the high availability of control node. When one of the controller failed to connect the network, which breaks down the Openstack services on this node. These Openstack service should able to be accessed by other controller nodes, and the services on failed controller node should be isolated
	OPNFV_YARDSTICK_TC051 This test case will verify the high availability of control node. When the CPU usage of a specified controller node is stressed to 100%, which breaks down the Openstack services on this node. These Openstack service should able to be accessed by other controller nodes, and the services on failed controller node should be isolated
	OPNFV_YARDSTICK_TC052 This test case will verify the high availability of control node. When the disk I/O of a specified disk is blocked, which breaks down the Openstack services on this node. Read and write services should still be accessed by other controller nodes, and the services on failed controller node should be isolated
	OPNFV_YARDSTICK_TC053 This test case will verify the high availability of the load balance service(current is HAProxy) that supports OpenStack on controller node. When the load balance service of a specified controller node is killed, whether other load balancers on other controller nodes will work, and whether the controller node will restart the load balancer are checked
	OPNFV_YARDSTICK_TC054 This test case will verify the high availability for virtual ip in the environment. When master node of virtual ip is abnormally shutdown, connection to virtual ip and the services binded to the virtual IP it should be OK

IPv6

Test Case	Test Case Group	Description	Gerrit References
IPv6 IPAM for an instance	IPv6	Verify DHCPv6 (stateful & stateless), SLAAC	https://gerrit.opnfv.org/gerrit/#/c/20719/ https://gerrit.opnfv.org/gerrit/#/c/23843/
Security rules	IPv6	IPv6 security rules for Neutron
IPv6 routing	IPv6	using switch/router, IPv6 service VM etc.
OPNFV_YARDSTICK_TC027_IPv6 connectivity To do a basic verification that IPv6 connectivity is within acceptable boundaries when ipv6 packets travel between hosts located on same or different compute blades.

VPN

Test Case	Test Case Group	Description
VPN operations	VPN CRUD	Create VPN instance and check command return code Check if VPN instance is present Create IETF VM interface and check return code Verify IETF VM interface Create VPN interface for IETF interface Verify VPN interface Verify FIB entry after create Delete VM VPN interface Verify after deleting VM VPN interface Delete VPN instance Verify after deleting VPN instance Delete VM IETF interface Verify after deleting VM IETF interface Verify FIB entry after delete
VPN operations	BGP VPN CRUD	Create BGPVPN Create BGPVPN with malformatted route target (e.g. ASN:NN) should fail. Create BGPVPN with invalid route target (e.g. 65536:0) should fail. Getting the VPN list works without producing an error. Updating an existing BGPVPN works. Displaying parameters of an existing BGPVPN works. Deleting a BGPVPN works. Associating an existing BGPVPN with a Neutron network works Getting the associated Neutron network works. Deleting the network association works. Associating an existing BGPVPN with a Neutron Router works Getting the associated Neutron router works. Deleting the router association works
Subnet connectivity	VPN DC	VPN provides connectivity between subnets. different Neutron subnets cannot reach each other unless they are connected to the same Router.
Tenant separation	VPN DC	VPNs enable using the same IP address ranges in different VPNs, which is an important feature for tenant separation in the DC. This test verifies if the correct VM is reached under a given IP address when IP addresses are used multiple times in the same DC.
Data Center Gateway integration	VPN Multisite	Data Center Gateway integration, check if routes are correctly exchanged between SDN controller and datacenter gateway
Inter-DC communication	VPN Multisite	Inter-DC communication, Check if communication between two data centers connected through an MPLS backbone works.

Fault Management(Doctor)

Test Case	Test Case Group	Description
Relaunch instance ?	Fault management	When an instance fails, can the platform restart the instance automatically?
Scale-up ?	Fault management	Can a new instance of a given type be launched when a certain load is hit?
Subscribe to a fault event	Fault management	Listen for an instance fault , and verify that when the instance is killed we hear about it

KVM

ID	Test Case	Test Case Group	Description	Status	Gerrit References
1	evaluate the KVM virtualization capability

Parser

ID	Test Case	Test Case Group	Description	Status	Gerrit References
1	verify the function of Yang-to-Tosca in Parser

virtual Traffic Classifier

ID	Test Case	Test Case Group	Description	Status	Gerrit References
1	To verify that a newly instantiated vTC is ‘alive’ and functional and its instantiation is correctly supported by the infrastructure
2	To verify that a newly instantiated vTC is ‘alive’ and functional and its instantiation is correctly supported by the infrastructure in the presence of noisy neighbours.

Copper

ID	Test Case	Description
1	policy management
	DMZ Placement	Pause VMs running in a DMZ zone, if the image running is not allowed to run in that zone
	SMTP Ingress	Identify VMs that have TCP port 25 open for ingress
	Reserved Subnet	Delete any subnets created in restricted network ranges

Promise

ID	Test Case	Description
1	Resource reservation
	Capacity Management	Including add providers, update the capacity, and query the current capacity and utilization of a provider
	Reservation Management	allows a Consumer to request reservations for resource capacity (create, update, cancel, query reservations)
	Allocation Management	Create an instance of specified resource(s) for immediate use utilizing capacity from the pool and destroy an existing server instance and release it back to the pool

Multisite

ID	Test Case	Test Case Group	Description	Status	Gerrit References
1	multiple connected OpenStack deployments, of different sizes

SDN controller(ODL)

ID	Test Case	Test Case Group	Description	Status	Gerrit References
1

SDN controller(ONOS)

ID	Test Case	Test Case Group	Description	Status	Gerrit References
1

Open source VNF running on NFVI

ID	Test Case	Test Case Group	Description	Status	Gerrit References
1

Space shortcuts

Page tree