scenarios	installer	cvp.0.1.0	cvp.0.2.0	cvp.0.3.0	cvp.0.4.0	cvp.0.5.0	cvp.0.6.0
Apex Scenarios Currently Test Against
os-nosdn-nofeature-ha	Apex					https://build.opnfv.org/ci/view/dovetail/job/dovetail-apex-huawei-pod4-proposed_tests-danube/4/console PASS/total defcore: 204/205 HA: 7/8 vping: 2/2 ipv6: 25/25 tempest: 4/6 SDNVPN: 0/4 no SDN feature	https://build.opnfv.org/ci/view/dovetail/job/dovetail-apex-huawei-pod4-proposed_tests-danube/199/consoleFull PASS/total defcore: 205/205 HA: 7/8 vping: 2/2 ipv6: 25/25 tempest: 5/5 SDNVPN: 0/4 no SDN feature
os-odl-bgpvpn-ha	Apex
Compass Scenarios Currently Test Against
os-nosdn-nofeature-ha	Compass	https://build.opnfv.org/ci/view/dovetail/job/dovetail-compass-virtual-proposed_tests-danube/176/consoleFull PASS/total defcore: 204/205 HA: 7/8 tc007, SLA not passed ipv6: 25/25 SDNVPN: 0/4 no SDN feature	https://build.opnfv.org/ci/view/dovetail/job/dovetail-compass-baremetal-proposed_tests-danube/157/consoleFull PASS/total defcore: 204/205 HA: 8/8 ipv6: 25/25 SDNVPN: 0/4 no SDN feature	https://build.opnfv.org/ci/view/dovetail/job/dovetail-compass-baremetal-proposed_tests-danube/196/consoleFull PASS/total defcore: 204/205 HA: 8/8 ipv6: 25/25 SDNVPN: 0/4 no SDN feature	https://build.opnfv.org/ci/view/dovetail/job/dovetail-compass-virtual-proposed_tests-danube/335/consoleFull PASS/total defcore: 204/205 HA: 8/8 vping: 2/2 ipv6: 25/25 SDNVPN: 0/4 no SDN feature resiliency: 1/1	https://build.opnfv.org/ci/view/dovetail/job/dovetail-compass-baremetal-proposed_tests-danube/288/consoleFull PASS/total defcore: 205/205 HA: 8/8 vping: 2/2 ipv6: 25/25 tempest: 3/6 SDNVPN: 0/4 no SDN feature	https://build.opnfv.org/ci/view/dovetail/job/dovetail-compass-baremetal-proposed_tests-danube/331/consoleFull PASS/total defcore: 205/205 HA: 8/8 vping: 2/2 ipv6: 25/25 tempest: 5/5 SDNVPN: 0/4 no SDN feature
os-odl_l2-nofeature-ha	Compass		https://build.opnfv.org/ci/view/dovetail/job/dovetail-compass-baremetal-proposed_tests-danube/163/consoleFull PASS/total defcore: 204/205 HA: 8/8 ipv6: 25/25 SDNVPN: 0/4 no SDN feature	https://build.opnfv.org/ci/view/dovetail/job/dovetail-compass-baremetal-proposed_tests-danube/191/consoleFull PASS/total defcore: 204/205 HA: 8/8 ipv6: 17/25 SDNVPN: 0/4 no SDN feature	https://build.opnfv.org/ci/view/dovetail/job/dovetail-compass-virtual-proposed_tests-danube/309/consoleFull PASS/total defcore: 204/205 HA: 7/8 vping: 2/2 ipv6: 25/25 SDNVPN: 0/4 no SDN feature resiliency: 1/1	https://build.opnfv.org/ci/view/dovetail/job/dovetail-compass-baremetal-proposed_tests-danube/304/consoleFull PASS/total defcore: 205/205 HA: 8/8 vping: 2/2 ipv6: 24/25 tempest: 2/6 SDNVPN: 0/4 no SDN feature	https://build.opnfv.org/ci/view/dovetail/job/dovetail-compass-baremetal-proposed_tests-danube/335/consoleFull PASS/total defcore: 205/205 HA: 8/8 vping: 2/2 ipv6: 25/25 tempest: 4/5 (Failed on testing port's admin state) SDNVPN: 0/4 no SDN feature
os-odl_l3-nofeature-ha	Compass	https://build.opnfv.org/ci/view/dovetail/job/dovetail-compass-virtual-proposed_tests-danube/178/consoleFull PASS/total defcore: 204/205 HA: 7/8 tc007, SLA not passed ipv6: 17/25 tc018-tc025 fails SDNVPN: 0/4 no SDN feature	https://build.opnfv.org/ci/view/dovetail/job/dovetail-compass-baremetal-proposed_tests-danube/159/consoleFull PASS/total defcore: 204/205 HA: 8/8 ipv6: 17/25 tc018-tc025 fails SDNVPN: 0/4 no SDN feature	https://build.opnfv.org/ci/view/dovetail/job/dovetail-compass-baremetal-proposed_tests-danube/198/consoleFull PASS/total defcore: 204/205 HA: 8/8 ipv6: 17/25 tc018-tc025 fails SDNVPN: 0/4 no SDN feature	https://build.opnfv.org/ci/view/dovetail/job/dovetail-compass-baremetal-proposed_tests-danube/243/consoleFull PASS/total defcore: 204/205 HA: 8/8 vping: 2/2 ipv6: 17/25 tc018-tc025 fails SDNVPN: 0/4 no SDN feature resiliency: 0/1	https://build.opnfv.org/ci/view/dovetail/job/dovetail-compass-baremetal-proposed_tests-danube/311/consoleFull PASS/total defcore: 205/205 HA: 8/8 vping: 2/2 ipv6: 17/25 tc018-tc025 fails tempest: 2/6 SDNVPN: 0/4 no SDN feature	https://build.opnfv.org/ci/view/dovetail/job/dovetail-compass-baremetal-proposed_tests-danube/326/consoleFull PASS/total defcore: 205/205 HA: 8/8 vping: 2/2 ipv6: 17/25 tc018-tc025 fails tempest: 3/5 SDNVPN: 0/4 no SDN feature
Fuel Scenarios Currently Test Against
os-nosdn-nofeature-ha	Fuel (zte-pod1)						https://build.opnfv.org/ci/view/dovetail/job/dovetail-fuel-zte-pod1-proposed_tests-danube/2/consoleFull PASS/total defcore: 205/205 HA: 8/8 vping: 2/2 ipv6: 25/25 tempest: 4/5 SDNVPN: 0/4 no SDN feature
os-odl_l2-bgpvpn-ha	Fuel	https://build.opnfv.org/ci/view/dovetail/job/dovetail-fuel-baremetal-proposed_tests-danube/45/consoleFull PASS/total defcore: 205/205 HA: 8/8 ipv6: 17/25 tc018-tc025 fails SDNVPN: 3/4 sdnvpn.tc001 "	https://build.opnfv.org/ci/view/dovetail/job/dovetail-fuel-baremetal-proposed_tests-danube/52/console PASS/total defcore: 205/205 HA: 8/8 ipv6: 17/25 tc018-tc025 fails SDNVPN: 4/4	https://build.opnfv.org/ci/view/dovetail/job/dovetail-fuel-baremetal-proposed_tests-danube/71/consoleFull PASS/total defcore: 205/205 HA: 8/8 ipv6: 17/25 tc018-tc025 fails SDNVPN: 4/4
Other Scenarios
os-nosdn-kvm-ha	Apex
os-odl_l3-fdio-ha	Apex
os-onos-nofeature-ha Don't test against this scenario since Compass will not release this scenario anymore.	Compass	~~https://build.opnfv.org/ci/view/dovetail/job/dovetail-compass-virtual-proposed_tests-danube/177/consoleFull~~ ~~PASS/total~~ ~~defcore: 150/171~~ ~~HA: 7/8 tc007, SLA not passed~~ ~~ipv6: 17/25 tc018-tc025 fails~~ ~~SDNVPN: 0/4 no SDN feature~~	~~https://build.opnfv.org/ci/view/dovetail/job/dovetail-compass-baremetal-proposed_tests-danube/160/consoleFull~~ ~~PASS/total~~ ~~defcore: 198/205~~ ~~HA: 8/8~~ ~~ipv6: 17/25 tc018-tc025 fails~~ ~~SDNVPN: 0/4 no SDN feature~~	~~https://build.opnfv.org/ci/view/dovetail/job/dovetail-compass-baremetal-proposed_tests-danube/195/consoleFull~~ ~~PASS/total~~ ~~defcore: 198/205~~ ~~HA: 8/8~~ ~~ipv6: 17/25 tc018-tc025 fails~~ ~~SDNVPN: 0/4 no SDN feature~~	~~https://build.opnfv.org/ci/view/dovetail/job/dovetail-compass-virtual-proposed_tests-danube/307/consoleFull~~ ~~PASS/total~~ ~~defcore: 198/205~~ ~~HA: 7/8~~ ~~vping: 2/2~~ ~~ipv6: 17/25 tc018-tc025 fails~~ ~~SDNVPN: 0/4 no SDN feature~~ ~~resiliency: 0/1~~	~~https://build.opnfv.org/ci/view/dovetail/job/dovetail-compass-baremetal-proposed_tests-danube/312/consoleFull~~ ~~PASS/total~~ ~~defcore: 198/205~~ ~~HA: 5/8~~ ~~vping: 2/2~~ ~~ipv6: 17/25 tc018-tc025 fails~~ ~~tempest: 1/6~~ ~~SDNVPN: 0/4 no SDN feature~~
os-odl_l2-nofeature-ha	Fuel(ZTE-POD1)
os-nosdn-ovs-ha	Fuel
os-odl_l3-nofeature-ha	Fuel
os-nosdn-kvm_ovs_dpdk-ha	Fuel

cvpcvp.0.5.0 Tempest Test Cases Results on Compass Danube

...

Dovetail Test Cases Run Results for Apex, Compass and Fuel

4. Known Issues List

Issue description	Test case dovetail.ha.tc008 failed due to the HA settings on Apex.
Reason	This test case will fail on Apex danube SUT due to haproxy processes outage time is larger than 30s or the openstack service outage time is larger than 5s after killing all haproxy processes in one controller node. The failure is be caused by the HA setting of the SUT. It uses pacemaker to realize HA, and there are two settings that affect this test case. The haproxy. The default monitor interval time of it is 60 seconds, which means it will check haproxy every 60 seconds. haproxy processes will recover after it detects the failure. This interval time may cause the recover time to be more than 30 seconds sometimes. The float of virtual ip. The default monitor interval time of it is 10 seconds and the monitor timeout of it is 20 seconds. This means it will check the virtual ip every 10 seconds and float it to another controller node after 20 seconds when it detects the failure. All this make the openstack services recover time always larger than 5 seconds.
Affect	All SUTs use pacemaker and the settings of the recover time is too long.
Status	Suggest to configure SUT' uses enlarge test case’s process outage time and service outage time to 60s.
References	Test case specification http://artifacts.opnfv.org/dovetail/docs/testing_user_testspecification_highavailability/index.html#test-case-8-controller-load-balance-as-a-service-high-availability

Issue description	Test case failed when using ODL as the network management provider. tempest.scenario.test_security_groups_basic_ops.TestSecurityGroupsBasicOps.test_cross_tenant_traffic
Reason	Refer to [1], this test case will fail on test assertion 1. Conntrack isn't enabled with odl legacy netvirt by default. With enabling conntracking, this test case passes. This is a bug of neworking-odl and the bug report link is [2]. But this bug won’t fix, because conntrack requires relatively new version of odl and linux kernel. It is unstable with odl legacy netvirt.
Affect	All SUTs use odl and disable conntrack.
Status	Suggest testers enable conntrack.
References	test case specification http://artifacts.opnfv.org/dovetail/docs/testing_user_testspecification_securitygroup/index.html#test-case-2-test-security-group-cross-tenant-traffic bug report https://bugs.launchpad.net/networking-odl/+bug/1643033

Issue description	Test cases failed when using ODL as the network management provider. tempest.scenario.test_network_basic_ops.TestNetworkBasicOps.test_update_instance_port_admin_state tempest.scenario.test_network_basic_ops.TestNetworkBasicOps.test_update_router_admin_state
Reason	Refer to [1], these two test cases will fail on test assertion 3 and test assertion 2 respectively. The ping operation will still be successful after setting port’s/router’s admin_state_up attribute to be false. This is a bug of ODL Boron release and fixed with Carbon release.
Affect	All SUTs use odl Boron or before.
Status	None
References	test case specification http://artifacts.opnfv.org/dovetail/docs/testing_user_testspecification_dynamicnetwork/index.html#dynamic-network-runtime-operations-test-specification bug report https://bugs.opendaylight.org/show_bug.cgi?id=6718 https://bugs.opendaylight.org/show_bug.cgi?id=4543

Issue description	Test cases failed when using ODL as the network management provider. ipv6.tc018 to ipv6.tc025
Reason	All these test cases try to ping6 to the router internal interface. In ODL Boron release, it supports IPv6 IPAM (IPAddress Assignment), IPv6 Security Groups and IPv6 east-west routing. Support for IPv6 north-south communication and ping6 support for neutron router-interface is currently under progress.
Affect	All SUTs use odl Boron or before.
Status	None
References	test case specification http://artifacts.opnfv.org/dovetail/docs/testing_user_testspecification_ipv6/index.html

Issue description	Test case is unstable on Fuel tempest.scenario.test_security_groups_basic_ops.TestSecurityGroupsBasicOps.test_port_security_disable_security_group
Reason	During out test, this test case sometimes passes but sometimes fails on Fuel.
Affect	unknonw
Status	None
References	test case specification http://artifacts.opnfv.org/dovetail/docs/testing_user_testspecification_securitygroup/index.html#test-case-5-test-port-security-disable-security-group

...

company	cloudOS	feedbacks
ZTE	ZTE TECS
Wind River	Wind River Titanium 3 based on Mitaka with AVS
Huawei	FS
Redhat
BII	OPNFV Distr

...

6. Running in the OPNFV plugfest(orange)

hardware	installer	scenarios	dovetail test cases	results	owner
Nokia hardware	apex	os-nosdn-nofeature-ha	defcore and IPV6	only one faill: (failing test): tempest.api.compute.volumes.test_attach_volume.Attach VolumeTestJSON.test_attach_detach_volume FAIL Note: This can be solved by manually set "volume_device_name" in refstack configuration file tempest.conf according to the platform, it's a hard-coded configuration in functest, actually. volume_device_name = vdb or vdc or others	Rautakumpu, Mika
Huawei hardware	compass	os-nosdn-nofeature-ha	defcore and IPV6	all pass	hongbo tian
lenovo	Joid	os-nosdn-nofeature-ha	defcore and IPV6	all pass	Narinder Gupta
lenovo		Wind River Titanium 3 based on Mitaka with AVS	defcore and IPV6	all pass	Raineri, Eddy

...

Space shortcuts

Page tree

Versions Compared

Old Version 70

New Version 71

Key

4. Known Issues List

6. Running in the OPNFV plugfest(orange)

Space shortcuts

Page tree

Page History

Versions Compared

Old Version 70

New Version 71

Key

4. Known Issues List

6. Running in the OPNFV plugfest(orange)