2018-06-25
| Installer | Scenario | stress.ping | tempest.compute | tempest.identity_v2 | tempest.identity_v3 | tempest.image | tempest.network_api | tempest.volume | tempest.trunk_ports | security.patrole | ha.controller_restart | ha.rabbitmq | ha.database | ha.neutron_l3_agent | tempest.network_scenario | vnf.vEPC | vnf.vIMS | tempest.bgpvpn | snaps.smoke |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Apex | stable/fraser | FAIL | PASS | FAIL - only v3 used skip mechanism needed or just part of reviewer instructions (does not need to be automated) | PASS | PASS | PASS | PASS | PASS | PASS | FAIL IPMI info missing | PASS | PASS | PASS | PASS | FAIL passes on Apex daily loop | PASS | not supported | FAIL test_volume_type_with_encryption test_volume_type_with_qos_and_encryption passes on Apex daily CI loop |
stable/fraser | FAIL | FAIL | FAIL - only v3 used skip mechanism needed or just part of reviewer instructions (does not need to be automated) | FAIL | FAIL | FAIL | FAIL | FAIL IPMI info missing | PASS | PASS | not supported: l3 agent not installed | FAIL | FAIL | FAIL | FAIL | FAIL | |||
master | FAIL | PASS | not supported | PASS | PASS | PASS | PASS | PASS | PASS | FAIL IPMI info missing | PASS | PASS | PASS | PASS | PASS | FAIL VM in error state: "no valid host found" passes on Apex daily CI loop | not supported | FAIL | |
| Compass | stable/fraser | PASS create stack rate 5/5 | PASS | SKIP | PASS | PASS | PASS | PASS | PASS skip: 9 pass: 43 | PASS skip: 3 (dvr is not enabled) pass: 132 test_create_distributed_router ... skiptest_show_distributed_router ... skiptest_update_distributed_router ... skip | PASS | PASS | PASS | FAIL SLA failure: 9.023325 > 5.000000 | FAIL fail: 5 No IPv4 addresses found | PASS | PASS | not supported | PASS |
stable/fraser | FAIL failed to create stacks | PASS | SKIP | PASS | PASS | FAIL fail: 3 pass: 17 | PASS | PASS skip: 9 pass: 43 | PASS skip: 5 pass: 130 | PASS | PASS | PASS | not supported: l3 agent not installed | FAIL fail: 3 pass: 17 | PASS | PASS | not supported | FAIL fail: 2 pass: 117 | |
| Fuel (x86) | master | FAIL | FAIL | FAIL - only v3 used skip mechanism needed or just part of reviewer instructions (does not need to be automated) | PASS | PASS | FAIL - Success: 18
- Skipped: 0
- Expected failures: 0
- Unexpected success: 0
- Failures: 1 test_hotplug_nic ... fail | PASS | PASS skip: 9 pass: 43 | PASS skip: 3 (dvr is not enabled) pass: 132 test_create_distributed_router ... skiptest_show_distributed_router ... skiptest_update_distributed_router ... skip | PASS | FAIL rabbitmq located in msg0x VMs | FAIL database located in dbs0x VMs | FAIL l3-agent in gtw0x VMs | FAIL - Success: 18
- Skipped: 0
- Expected failures: 0
- Unexpected success: 0
- Failures: 1 test_hotplug_nic ... fail | FAIL passes on FUEL daily CI loop | FAIL deployment clearwater-opnfv timed out | not supported | PASS |
| Installer | Scenario | vping_ssh | vping_userdata | tempest.osinterop | tempest.ipv6 | tempest.multi_node_scheduling | tempest.security | tempest.vm_lifecycle | ha.nova_api | ha.neutron_server | ha.keystone | ha.glance_api | ha.cinder_api | ha.cpu_load | ha.disk_load | ha.haproxy | sdnvpn.subnet_connectivity | sdnvpn.tenant_separation | sdnvpn.router_assoc | sdnvpn.router_assoc_fip |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Apex | stable/fraser | PASS | FAIL passes on Apex daily loop | FAIL passes on Apex daily CI loop | FAIL | FAIL | FAIL | FAIL | PASS | FAIL | PASS | PASS | FAIL | PASS | FAIL IPMI info missing | PASS | not supported | not supported | not supported | not supported |
| Fuel | nosdn-nofeature | PASS | PASS | FAIL | FAIL | FAIL | FAIL | FAIL | PASS | PASS | FAIL | PASS | FAIL | PASS | PASS | PASS | not supported | not supported | not supported | not supported |
Historical results below
2018-05-23 - limited results due to failed CI jobs
| Installer | Scenario | vnf.vIMS | snaps.smoke | stress.ping | tempest.compute | tempest.identity_v2 | tempest.identity_v3 | tempest.image | tempest.network | tempest.volume | tempest.trunk_ports | tempest.bgpvpn | security.patrole | ha.controller_restart | ha.rabbitmq | ha.neutron_l3_agent | ha.database |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Apex | nosdn-nofeature | FAIL Exception: Unable to create a number | PASS | FAIL | PASS | FAIL - only v3 used skip mechanism needed | PASS | PASS | FAIL - Success: 19 - Skipped: 0 - Expected failures: 0 - Unexpected success: 0 - Failures: 1 test_mtu_sized_frames ... fail | PASS | PASS | not supported | PASS | FAIL | FAIL SLA failure: 31.512733 > 30.000000 | FAIL stack create failed inner exception: file not exist | PASS |
| odl | FAIL | FAIL | PASS | FAIL | PASS | PASS | FAIL | PASS | PASS | FAIL | FAIL | FAIL | FAIL | l3 agent not installed | PASS | ||
| Compass | nosdn-nofeature | ||||||||||||||||
| odl | FAIL | FAIL fail: 2 pass: 117 | FAIL failed to create stacks | PASS | SKIP | PASS | PASS | FAIL fail: 3 pass: 17 | PASS | PASS skip: 9 pass: 43 | FAIL No .testr.conf file was found | PASS skip: 5 pass: 130 | PASS | FAIL No rabbitmq-server process | FAIL No neutron-la-agent process | PASS | |
| Fuel (x86) | nosdn-nofeature | FAIL deployment clearwater-opnfv timed out | PASS | PASS | PASS | SKIP | PASS | PASS | FAIL fail: 1 pass: 19 test_mtu_sized_frames ... fail | PASS | PASS skip: 9 pass: 43 | FAIL No .testr.conf file was found | PASS | FAIL IPMI information missing | FAIL No rabbitmq-server process | FAIL failed to create stack with shade | FAIL No mysql process |
| Fuel (arm) |
2018-05-16
| Installer | Scenario | smoke.tc001 | smoke.tc002 | ha.tc009 | ha.tc010 | ha.tc011 | vnf.tc001 | stress.tc001 | sdnvpn.tc000 | security.patrole | tempest.neutron_trunk_ports |
|---|---|---|---|---|---|---|---|---|---|---|---|
| Apex | nosdn-nofeature | FAIL 0 failing tests 76 success 19 skipped: skip: Identity api v2 is not enabled skip: AccountQuotasTest skipped as swift => implement mechanism for specifying skippable tests | FAIL 1 failing test: PortConfigError: Cannot locate network | FAIL ValueError: context not found IPMI information missing? | FAIL missing ssh key in Yardstick | FAIL Unable to find network Env variables not passed correctly | FAIL failed to download file (temporary problem?) | FAIL TypeError: cannot concatenate 'str' => OS_TENANT_NAME not defined in OS credentials on Apex | -- not supported by scenario -- | PASS | -- not yet run -- |
| odl | FAIL 0 failing tests 76 success 19 skipped: skip: Identity api v2 is not enabled skip: AccountQuotasTest skipped as swift=> implement mechanism for specifying skippable tests | FAIL 1 failing test: PortConfigError: Cannot locate network | FAIL ValueError: context not found IPMI information missing? | FAIL missing ssh key in YardstickFIX https://gerrit.opnfv.org/gerrit/#/c/57401/ | FAIL Unable to find network Env variables not passed correctly | FAIL out of floating IPs | FAIL TypeError: cannot concatenate 'str' => OS_TENANT_NAME not defined in OS credentials on Apex | FAIL FIX: https://gerrit.opnfv.org/gerrit/#/c/57263/ | PASS | -- not yet run -- | |
| Compass | nosdn-nofeature | FAIL 1 failing tests 75 success 19 skipped fail: No IPv4 address found | FAIL 2 failing tests fail: Guest does not have a console available. (HTTP 404) fail: network_id are not the same | PASS | PASS | FAIL SLA failure: 10.010200 > 5.000000 | PASS (nosdn-nofeature) FAIL failed to download files | FAIL | --- | PASS 0 failing tests 132 success 3 skipped skip: dvr is not enabled | PASS 0 failing tests 43 success 9 skipped skip: vlan, vxlan, gre skip: Advanced image is required to run this test |
| odl | FAIL 0 failing tests 76 success 19 skipped | FAIL 1 failing tests fail: <snaps.domain.network.Router instance is not None | PASS | PASS | --- | FAIL failed to download files | FAIL No valid host was found | --- | FAIL | ||
| Fuel (x86) | nosdn-nofeature | FAIL 0 failing tests 76 success 19 skipped | PASS 119 success | FAIL IPMI information missing | FAIL Ssh is still | FAIL failed to create stack with shade | FAIL failed to download files | PASS | --- | PASS 135 success | PASS 0 failing tests 43 success 9 skipped skip: vlan, vxlan, gre skip: Advanced image is required to run this test |
| Fuel (arm) |
2018-05-09
| Installer | Scenario | smoke.tc001 | smoke.tc002 | ha.tc009 | ha.tc010 | ha.tc011 | vnf.tc001 | stress.tc001 | sdnvpn.tc000 | security.patrole | tempest.neutron_trunk_ports |
|---|---|---|---|---|---|---|---|---|---|---|---|
| Apex | FAIL 0 failing tests 76 success 19 skipped: skip: Identity api v2 is not enabled skip: AccountQuotasTest skipped as swift | FAIL PortConfigError: Cannot locate network | FAIL ValueError: context not found IPMI information missing | FAIL Ssh is still | FAIL Unable to find network Env variables not passed correctly? Dovetail needs to pass the env file. We (Xudan) have seen this before. | PASS (nofeature) ? (odl) | FAIL TypeError: cannot concatenate 'str' | FAIL Unknown test case or tier 'bgpvpn', or not supported by the given scenario 'unknown' | PASS | ||
| Compass | nosdn-nofeature | FAIL 1 failing tests 75 success 19 skipped fail: No IPv4 address found | FAIL 2 failing tests fail: Guest does not have a console available. (HTTP 404) fail: network_id are not the same | PASS | PASS | FAIL SLA failure: 10.010200 > 5.000000 | PASS (nosdn-nofeature) FAIL failed to download files | FAIL | --- | PASS 0 failing tests 132 success 3 skipped skip: dvr is not enabled | PASS 0 failing tests 43 success 9 skipped skip: vlan, vxlan, gre skip: Advanced image is required to run this test |
| odl | FAIL 0 failing tests 76 success 19 skipped | FAIL 1 failing tests fail: <snaps.domain.network.Router instance is not None | PASS | PASS | --- | FAIL failed to download files | FAIL No valid host was found | --- | FAIL | ||
| Fuel (x86) | nosdn-nofeature | FAIL 0 failing tests 76 success 19 skipped | PASS 119 success | FAIL IPMI information missing | FAIL Ssh is still | FAIL failed to create stack with shade | FAIL failed to download files | PASS | --- | PASS 135 success | PASS 0 failing tests 43 success 9 skipped skip: vlan, vxlan, gre skip: Advanced image is required to run this test |
| Fuel (arm) |