...
| ID | Type | Test Case | Description | Pre-condition & requirements | Status | Gerrit References |
|---|---|---|---|---|---|---|
| 1 | NFVICompute | Basic minimum scenario test | 1. Create image 2. Create keypair 3. Boot instance with keypair and get list of instances 4. Create volume and show list of volumes 5. Attach volume to instance and getlist of volumes 6. Add IP to instance 7. Create and add security group to instance 8. Check SSH connection to instance 9. Reboot instance 10. Check SSH connection to instance after reboot | VIM: Openstack Components: Keystone, Nova, Neutron, Glance, Cinder | ||
| 2 | Compute | OPNFV_YARDSTICK_TC010 | Measure the memory read latency for varying memory sizes and strides. Whole memory hierarchy is measured including all levels of cache. | Colorado Scenario:due to floating IP,not support odl_l2, bgnvpn VIM: Openstack Components: Keystone, Nova, Neutron, Glance, Cinder, Heat | ||
| 3 | Compute | OPNFV_YARDSTICK_TC012 | Measure the rate at which data can be read from and written to the memory (this includes all levels of memory). | Colorado Scenario: due to floating IP,not support odl_l2, bgnvpn VIM: Openstack Components: Keystone, Nova, Neutron, Glance, Cinder, Heat | ||
| 4 | Compute | OPNFV_YARDSTICK_TC014 | To evaluate the IaaS processing speed with regards to score of single cpu running and parallel running The purpose is also to be able to spot trends. Test results, graphs and similar shall be stored for comparison reasons and product evolution understanding between different OPNFV versions and/or configurations. | Colorado Scenario: due to floating IP,not support odl_l2, bgnvpn VIM: Openstack Components: Keystone, Nova, Neutron, Glance, Cinder, Heat | ||
| 5 | Compute | OPNFV_YARDSTICK_TC055 | To evaluate the IaaS compute capacity with regards to hardware specification, including number of cpus, number of cores, number of threads, available memory size and total cache size. Test results, graphs and similar shall be stored for comparison reasons and product evolution understanding between different OPNFV versions and/or configurations. | Colorado Scenario:due to floating IP,not support odl_l2, bgnvpn VIM: Openstack Components: Keystone, Nova, Neutron, Glance, Cinder, Heat | ||
| 6 | Compute | Aggregates basic ops | Creates an aggregate within an availability zone 1. Adds a host to the aggregate 2. Checks aggregate details 3. Updates aggregate's name 4. Removes host from aggregate 5. Deletes aggregate | VIM: Openstack Components: Keystone, Nova, Neutron, Glance, Cinder | ||
| 7 | Compute | Server advanced operations | This test case stresses some advanced server instance operations: * Resizing a volume-backed instance * Sequence suspend resume | VIM: Openstack Components: Keystone, Nova, Neutron, Glance, Cinder | ||
| 8 | Compute | Server basic operations | This smoke test case follows this basic set of operations: * Create a keypair for use in launching an instance * Create a security group to control network access in instance * Add simple permissive rules to the security group * Launch an instance * Perform ssh to instance * Verify metadata service * Verify metadata on config_drive * Terminate the instance | VIM: Openstack Components: Keystone, Nova, Neutron, Glance, Cinder | ||
| 9 | Compute | Server multinode | This is a set of tests specific to multinode testing | VIM: Openstack Components: Keystone, Nova, Neutron, Glance, Cinder | ||
| 10 | Compute | Shelve instance | The following is the scenario outline: * boot an instance and create a timestamp file in it * shelve the instance * unshelve the instance * check the existence of the timestamp file in the unshelved instance | VIM: Openstack Components: Keystone, Nova, Neutron, Glance, Cinder | ||
| 11 | Compute | Snapshot pattern | The following is the outline: * boot an instance and create a timestamp file in it * snapshot the instance * boot a second instance from the snapshot * check the existence of the timestamp file in the second instance | VIM: Openstack Components: Keystone, Nova, Neutron, Glance, Cinder | ||
| 12 | Compute | Stamp pattern | This test is for snapshotting an instance/volume and attaching the volume created from snapshot to the instance booted from snapshot. The following is the scenario outline: 1. Boot an instance "instance1" 2. Create a volume "volume1" 3. Attach volume1 to instance1 4. Create a filesystem on volume1 5. Mount volume1 6. Create a file which timestamp is written in volume1 7. Unmount volume1 8. Detach volume1 from instance1 9. Get a snapshot "snapshot_from_volume" of volume1 10. Get a snapshot "snapshot_from_instance" of instance1 11. Boot an instance "instance2" from snapshot_from_instance 12. Create a volume "volume2" from snapshot_from_volume 13. Attach volume2 to instance2 14. Check the existence of a file which created at 6. in volume2 | VIM: Openstack Components: Keystone, Nova, Neutron, Glance, Cinder | ||
| 13 | Compute | Volume boot pattern | This test case attempts to reproduce the following steps: * Create in Cinder some bootable volume importing a image * Boot an instance from the bootable volume * Write content to the volume * Delete an instance and Boot a new instance from the volume * Check written content in the instance * Create a volume snapshot while the instance is running * Boot an additional instance from the new snapshot based volume * Check written content in the instance booted from snapshot | VIM: Openstack Components: Keystone, Nova, Neutron, Glance, Cinder |
NFVI
...
/network
| ID | Type | Test Case | Description | Pre-condition & requirements | Status | Gerrit References |
|---|---|---|---|---|---|---|
| 1 | Network | vPing | two vms setup in the same subnet and can ping each other (ssh, userdata, IPv4, IPv6(afraid not support now MatthewLi)) | VIM: Openstack Components: Keystone, Nova, Neutron, Glance | ||
| 2 | Network | vRouter | Two VMs in two different subnets, connected by a Neutron virtual router, can ping each other | VIM: Openstack Components: Keystone, Nova, Neutron, Glance | ||
| 3 | Network | Security rules | Verify that a security rule prevents a type of traffic, remove rule, verify that traffic passes, add it back, test that traffic is stopped | VIM: Openstack Components: Keystone, Nova, Neutron, Glance | ||
| 4 | Network | External routing | Verify that a VM can access hosts external to the VIM | VIM: Openstack Components: Keystone, Nova, Neutron, Glance | ||
| 5 | Network | ...fill out CRUD operations for "network/subnet/router/port" operations | VIM: Openstack Components: Keystone, Nova, Neutron, Glance | |||
| 6 | Network | Network advanced server ops | Check VM connectivity after some advanced instance operations executed: * Stop/Start an instance * Reboot an instance * Rebuild an instance * Pause/Unpause an instance * Suspend/Resume an instance * Resize an instance | VIM: Openstack Components: Keystone, Nova, Neutron, Glance, Cinder | ||
| 7 | Network | Network basic operations | This smoke test suite assumes that Nova has been configured to boot VM's with Neutron-managed networking, and attempts to verify network connectivity as follows: There are presumed to be two types of networks: tenant and public. A tenant network may or may not be reachable from the Tempest host. A public network is assumed to be reachable from the Tempest host, and it should be possible to associate a public ('floating') IP address with a tenant ('fixed') IP address to facilitate external connectivity to a potentially unroutable tenant IP address. | VIM: Openstack Components: Keystone, Nova, Neutron, Glance, Cinder | ||
| 8 | Network | Network v6 | 1. Create network with subnets: 1.1. one IPv4 and 1.2. one or more IPv6 in a given address mode 2. Boot 2 VMs on this network 3. Allocate and assign 2 FIP4 4. Check that vNICs of all VMs gets all addresses actually assigned 5. Each VM will ping the other's v4 private address 6. If ping6 available in VM, each VM will ping all of the other's v6 addresses as well as the router's | VIM: Openstack Components: Keystone, Nova, Neutron, Glance, Cinder | ||
| 9 | Network | Security groups basic ops | This test case assumes that Nova has been configured to boot VM's with Neutron-managed networking, and attempts to verify cross tenant connectivity as follows ssh: in order to overcome "ip namespace", each tenant has an "access point" VM with floating-ip open to incoming ssh connection allowing network commands (ping/ssh) to be executed from within the tenant-network-namespace Tempest host performs key-based authentication to the ssh server via floating IP address connectivity test is done by pinging destination server via source server ssh connection. success - ping returns failure - ping_timeout reached multi-node: Multi-Node mode is enabled when CONF.compute.min_compute_nodes > 1. Tests connectivity between servers on different compute nodes. When enabled, test will boot each new server to different compute nodes. | VIM: Openstack Components: Keystone, Nova, Neutron, Glance, Cinder |
NFVI
...
/storage
| ID | Type | Test Case | Description | Pre-condition & requirements | Status | Gerrit References |
|---|---|---|---|---|---|---|
| 1 | Storage | yardstick_tc005 | Fio test is invoked in a host VM on a compute blade, a job file as well as parameters are passed to fio and fio will start doing what the job file tells it to do. | Colorado Scenario: due to floating IP,not support odl_l2, bgnvpn VIM: Openstack Components: Keystone, Nova, Neutron, Glance, Cinder, Heat | ||
| 2 | Storage | Encrypted Cinder volumes | This test is for verifying the functionality of encrypted cinder volumes. For both LUKS and cryptsetup encryption types, this test performs the following: * Creates an image * Boots an instance from the image * Creates an encryption type (as admin) * Creates a volume of that encryption type (as a regular user) * Attaches and detaches the encrypted volume to the instance | VIM: Openstack Components: Keystone, Nova, Neutron, Glance, Cinder |
High Availability (HA)
| ID | Type | Test Case | Description | Pre-condition & requirements | Status | Gerrit References |
|---|---|---|---|---|---|---|
| 1 | HA | OPNFV_YARDSTICK_TC019 | This test case will verify the high availability of the service provided by OpenStack (nova-api) on control node. | VIM: Openstack Components: Keystone, Nova, Neutron, Glance, Cinder SDN: not support SDN Sepcial requirements: 3 control nodes with HA | ||
| 2 | HA | OPNFV_YARDSTICK_TC045 | This test case will verify the high availability of the network service provided by OpenStack (neutron-server) on control node | VIM: Openstack Components: Keystone, Nova, Neutron, Glance, Cinder SDN: not support SDN Sepcial requirements: 3 control nodes with HA | ||
| 3 | HA | OPNFV_YARDSTICK_TC046 | This test case will verify the high availability of the user service provided by OpenStack (keystone) on control node | VIM: Openstack Components: Keystone, Nova, Neutron, Glance, Cinder | ||
| 4 | HA | OPNFV_YARDSTICK_TC047 | This test case will verify the high availability of the image service provided by OpenStack (glance-api) on control node | VIM: Openstack Components: Keystone, Nova, Neutron, Glance, Cinder | ||
| 5 | HA | OPNFV_YARDSTICK_TC048 | This test case will verify the high availability of the volume service provided by OpenStack (cinder-api) on control node | VIM: Openstack Components: Keystone, Nova, Neutron, Glance, Cinder | ||
| 6 | HA | OPNFV_YARDSTICK_TC049 | This test case will verify the high availability of the storage service provided by OpenStack (swift-proxy) on control node | VIM: Openstack Components: Keystone, Nova, Neutron, Glance, Cinder Additional Components: Swift | ||
| 7 | HA | OPNFV_YARDSTICK_TC050 | This test case will verify the high availability of control node. When one of the controller failed to connect the network, which breaks down the Openstack services on this node. These Openstack service should able to be accessed by other controller nodes, and the services on failed controller node should be isolated | VIM: Openstack Components: Keystone, Nova, Neutron, Glance, Cinder SDN: not support SDN Sepcial requirements: 3 control nodes with HA | ||
| 8 | HA | OPNFV_YARDSTICK_TC051 | This test case will verify the high availability of control node. When the CPU usage of a specified controller node is stressed to 100%, which breaks down the Openstack services on this node. These Openstack service should able to be accessed by other controller nodes, and the services on failed controller node should be isolated | VIM: Openstack Components: Keystone, Nova, Neutron, Glance, Cinder SDN: not support SDN Sepcial requirements: 3 control nodes with HA | ||
| 9 | HA | OPNFV_YARDSTICK_TC052 | This test case will verify the high availability of control node. When the disk I/O of a specified disk is blocked, which breaks down the Openstack services on this node. Read and write services should still be accessed by other controller nodes, and the services on failed controller node should be isolated | VIM: Openstack Components: Keystone, Nova, Neutron, Glance, Cinder SDN: not support SDN Sepcial requirements: 3 control nodes with HA | ||
| 10 | HA | OPNFV_YARDSTICK_TC053 | This test case will verify the high availability of the load balance service(current is HAProxy) that supports OpenStack on controller node. When the load balance service of a specified controller node is killed, whether other load balancers on other controller nodes will work, and whether the controller node will restart the load balancer are checked | VIM: Openstack Components: Keystone, Nova, Neutron, Glance, Cinder SDN: not support SDN Sepcial requirements: 3 control nodes with HA | ||
| 11 | HA | OPNFV_YARDSTICK_TC054 | This test case will verify the high availability for virtual ip in the environment. When master node of virtual ip is abnormally shutdown, connection to virtual ip and the services binded to the virtual IP it should be OK | VIM: Openstack Components: Keystone, Nova, Neutron, Glance, Cinder SDN: not support SDN Sepcial requirements: 3 control nodes with HA |
...
| ID | Type | Test Case | Description | Pre-condition & requirements | Status | Gerrit References |
|---|---|---|---|---|---|---|
| 1 | Multisite | Quota Management API | This testcase includes 9 functions: 1. Update global limit for a tenant 2. Get global limit for a tenant 3. A tenant can also get the global limit by himself 4. Get defaults limits 5. Get total usage for a tenant 6. A tenant can also get the total usage by himself 7. On demand quota sync 8. Delete specific global limit for a tenant 9. Delete all kingbird global limit for a tenant | Colorado Scenario: os-nosdn-multisite-ha VIM: Openstack Components: Nova, Cinder, Neutron, KeyStone, Glance, ceilometer SDN: not support SDN Feature: no feature needed | ||
| 2 | Multisite | Quota Class API | This testcase includes 3 functions: 1. Update default quota class 2. Get default quota class 3. Delete default quota class | Colorado Scenario:os-nosdn-multisite-ha VIM: Openstack Components: Nova, Cinder, Neutron, KeyStone, Glance, ceilometer SDN: not support SDN Feature: no feature needed |
SDN controller(ODL)
| ID | Type | Test Case | Description | Pre-condition & requirements | Status | Gerrit References |
|---|---|---|---|---|---|---|
| 1 | ODL | restconf modules | verify Restconf is OK | Colorado Scenario: odl_l2 or odl_l3 VIM: Openstack Components: Neutron, ODL Feature: no feature needed | ||
| 2 | ODL | neutron reachability | Get the complete list of networks Get the complete list of subnets Get the complete list of ports | Colorado Scenario: odl_l2 or odl_l3 VIM: Openstack Components: Neutron, ODL Feature: no feature needed | ||
| 3 | ODL | neutron networks | Checking Network created in OpenStack are pushed to OpenDaylight | Colorado Scenario: odl_l2 or odl_l3 VIM: Openstack Components: Neutron, ODL Feature: no feature needed | ||
| 4 | ODL | neutron subnets | Checking Subnets created in OpenStack are pushed to OpenDaylight | Colorado Scenario: odl_l2 or odl_l3 VIM: Openstack Components: Neutron, ODL Feature: no feature needed | ||
| 5 | ODL | neutron ports | Checking Port created in OpenStack are pushed to OpenDaylight | Colorado Scenario: odl_l2 or odl_l3 VIM: Openstack Components: Neutron, ODL Feature: no feature needed | ||
| 6 | ODL | neutron delete ports | Checking Port deleted in OpenStack are deleted also in OpenDaylight | Colorado Scenario: odl_l2 or odl_l3 VIM: Openstack Components: Neutron, ODL Feature: no feature needed | ||
| 7 | ODL | neutron delete subnets | Checking Subnets deleted in OpenStack are deleted also in OpenDaylight | Colorado Scenario: odl_l2 or odl_l3 VIM: Openstack Components: Neutron, ODL Feature: no feature needed | ||
| 8 | ODL | neutron delete networks | Checking Network deleted in OpenStack are deleted also in OpenDaylight | Colorado Scenario: odl_l2 or odl_l3 VIM: Openstack Components: Neutron, ODL Feature: no feature needed |
...
Open source VNF running on NFVI
| ID | Type | Test Case | Description | Pre-condition & requirements | Status | Gerrit References |
|---|---|---|---|---|---|---|
| 1 | Open Source VNF | cloudify_ims | This test case deploys an OpenSource vIMS solution from Clearwater using the Cloudify orchestrator. It also runs some signaling traffic. | VIM: Openstack Components: Nova, Cinder, Neutron, KeyStone, Glance Special Requirements: Cloudify | ||
| 2 | Open Source VNF | orchestra_ims | ||||
| 3 | Open Source VNF | opera_ims | ||||
| 4 | Open Source VNF | vPE? | ||||
| 5 | Open Source VNF | vPE perfo? | ||||
| 6 | Open Source VNF | vEPC, vBBU, RRH, vUE |