====== Brahmaputra Summary ====== Please note that you may have a dynamic view per [[http://testresults.opnfv.org/proto/index2.html|installer]], [[http://testresults.opnfv.org/proto/index-pod.html|pod]] or [[http://testresults.opnfv.org/proto/index-scenario.html|scenario]] New Functest [[http://testresults.opnfv.org/dashboard/|dashboard]] look. this table reflects the status but it is not dynamically up to date as CI never stops.. We consider hereafter only baremetal scenario. Icon description: {{:weather-clear.png?32}} : 4+ consecutive runs passing the success criteria\\ {{:weather-few-clouds.png?32}} : <4 successful consecutive runs but passing the success criteria\\ {{:weather-overcast.png?32}} : close to pass the success criteria\\ {{:weather-storm.png?32}} : 0% success, not passing\\ === Fuel === POD: ericsson-pod2 \\ source: https://build.opnfv.org/ci/view/functest/job/functest-fuel-ericsson-pod2-daily-brahmaputra/ Scenario ready for Release: * nosdn * old_l2 * onos * nosdn-ovs odl_l3 blocked by vPing bgpvpn not mature ^ Scenario ^ VIM |||^ Controllers ||^ Features ||^ Comments ^ | | vPing SSH | vPing userdata | Tempest [1] | Rally | ODL | ONOS | OVNO | PROMISE | vIMS | SDNVPN | | | nosdn-nofeature | {{:weather-clear.png?32}} | {{:weather-clear.png?32}} | {{:weather-few-clouds.png?32}} | {{:weather-clear.png?32}} | N.R | N.R | N.R | {{:weather-clear.png?32}} | {{:weather-few-clouds.png?32}} | N.R | criteria reached on the 18/2 \\ Tempest rate 89% but some errors can be explained by config so can be removed from the equation leading to > 90% \\ vIMS suite #37, the previous and following ones were OK | | odl_l3-ha | {{:weather-few-clouds.png?32}} [3] | {{:weather-clear.png?32}} | {{:weather-overcast.png?32}} | {{:weather-clear.png?32}} | {{:weather-clear.png?32}} | N.R | N.R | {{:weather-clear.png?32}} | {{:weather-clear.png?32}} | N.R | Tempest success rate 83% | | odl_l2-ha | {{:weather-clear.png?32}} | {{:weather-clear.png?32}} | {{:weather-few-clouds.png?32}} | {{:weather-clear.png?32}} | {{:weather-clear.png?32}} | N.R | N.R | {{:weather-clear.png?32}} | {{:weather-clear.png?32}} | N.R | Tempest success rate 83% but errors mainly in test_server_action and orchestration.stacks. Some run exclude these tests and lead to > 90%, when included we have reproducible results | | odl_l2-bgpvpn-noha | {{:weather-few-clouds.png?32}} | {{:weather-few-clouds.png?32}} | {{:weather-overcast.png?32}} | {{:weather-storm.png?32}} | {{:weather-clear.png?32}} | N.R | N.R | {{:weather-few-clouds.png?32}} | {{:weather-storm.png?32}} | {{:weather-storm.png?32}} | | Run only once | onos-ha | {{:weather-clear.png?32}} [2] | N.R | {{:weather-few-clouds.png?32}} | {{:weather-clear.png?32}} | N.R | {{:weather-clear.png?32}} | N.R | {{:weather-clear.png?32}} | N.R | N.R | Tempest success rate 85% but test cases related to nova metadata shall be excluded leading to 90% | | nosdn-ovs-ha | {{:weather-clear.png?32}} | {{:weather-clear.png?32}} | {{:weather-overcast.png?32}} | {{:weather-clear.png?32}} | N.R | N.R | N.R | {{:weather-clear.png?32}} | {{:weather-clear.png?32}} | N.R | Tempest success rate 87% | | nosdn-kvm-ha | | | | | | | | | | | | | nosdn-ovs_kvm-ha | | | | | | | | | | | | [1]: Detected some errors (~7-8) possibly due to insufficient IPs in the public network ip range. Investigation in progress. Possible fix by Jonas: https://gerrit.opnfv.org/gerrit/#/c/10159/ [2]: It failed 2 times in the last 5 runs.. waiting for more runs to see if it is impracticable. Ashley proposed a patch to fix this and yardstick problems with SSH in this scenario [3]: last 2 runs it has failed \\ \\ === Apex === Scenario ready for Release: * old_l2 Doctor feature ready for the release we may assume that nosdn is working but never run in CI odl_l3 and onos blocked by vPing and Tempest success rate POD: opnfv-jump-1\\ source: https://build.opnfv.org/ci/view/functest/job/functest-apex-opnfv-jump-1-daily-brahmaputra/ \\ ^ Scenario ^ VIM |||^ Controllers ||^ Features |||^ Comments ^ | | vPing SSH | vPing userdata | Tempest | Rally | ODL | ONOS | OVNO | PROMISE | vIMS | Doctor | SDNVPN | | | odl_l3-ha | {{:weather-storm.png?32}} | {{:weather-storm.png?32}} | {{:weather-overcast.png?32}} | {{:weather-clear.png?32}} | {{:weather-few-clouds.png?32}} | N.R | N.R | N.R. | {{:weather-storm.png?32}} | {{:weather-few-clouds.png?32}} | N.R | Tempest rate not stable (from 70 to 90%), number of tests run varies a lot from one run to another \\ 3 ODL test cases partly failed | | odl_l2-ha | {{:weather-clear.png?32}} | {{:weather-clear.png?32}} | {{:weather-overcast.png?32}} | {{:weather-clear.png?32}} | {{:weather-few-clouds.png?32}} | N.R | N.R | N.R. | {{:weather-clear.png?32}} | {{:weather-clear.png?32}} | N.R. | Tempest rate not stable (from 60 to 90%), number of tests run varies a lot from one run to another \\ 3 ODL test cases partly failed | | onos-ha | {{:weather-storm.png?32}} | N.R [1] | {{:weather-clear.png?32}} | {{:weather-clear.png?32}} | N.R | {{:weather-clear.png?32}} | N.R | N.R. | N.R. | {{:weather-few-clouds.png?32}} | N.R. | Number of tempest test run varies but success rate always above 90% \\ ONOS suite run #54 failed due to connection to git (was OK in the 4 previous run) | | odl_l2-sfc | | | | | | | | | | | | | | ocl-nofeature-ha | | | | | | | | | | | | | [1] Nova metadata service is not working properly with ONOS yet. Maybe in SR1. \\ \\ === Compass === POD: huawei-us-deploy-bare-1 \\ source: https://build.opnfv.org/ci/view/functest/job/functest-compass-huawei-us-deploy-bare-1-daily-brahmaputra/ Scenarios ready for Release: * nosdn * odl_l2 * onos Ocl planned for SR1 ^ Scenario ^ VIM |||^ Controllers ||^ Features ||^ Comments ^ | | vPing SSH | vPing userdata | Tempest | Rally | ODL | ONOS | OVNO | PROMISE | vIMS | SDNVPN | | | nosdn-nofeature-ha | {{:weather-clear.png?32}} | {{:weather-clear.png?32}} | {{:weather-clear.png?32}} | {{:weather-clear.png?32}} | N.R | N.R | N.R | N.R | {{:weather-clear.png?32}} | N.R | success criteria reached 8/2 | | odl_l2-ha | {{:weather-clear.png?32}} | {{:weather-clear.png?32}} | {{:weather-clear.png?32}} | {{:weather-clear.png?32}} | {{:weather-clear.png?32}} | N.R | N.R | N.R | {{:weather-clear.png?32}} | N.R | success criteria reached 9/2 | | onos-ha | {{:weather-clear.png?32}} | N.R | {{:weather-clear.png?32}} [1] | {{:weather-clear.png?32}} | N.R | {{:weather-clear.png?32}} | N.R. | N.R | N.R | N.R | success criteria reached 4/2 | | ocl-nofeature-ha | {{:weather-storm.png?32}} | {{:weather-storm.png?32}} | {{:weather-storm.png?32}} | {{:weather-storm.png?32}} | N.R | N.R | {{:weather-storm.png?32}} | N.R | {{:weather-storm.png?32}} | N.R | planned for SR1, first run on 19/2 | [1]: There are 11 failed tests out of 210 which (we think) are related to nova metadata service not supported by onos yet. Some tests need to pass the ssh-key to the instances using that service. However, it passes the success criteria of >=90% \\ \\ === Joid === POD: orange-pod2 \\ source: https://build.opnfv.org/ci/view/functest/job/functest-joid-orange-pod2-daily-master/ Scenarios ready for Release: * odl_l2 nosdn scenario shall be OK but not run 4 times, will be added next time vIMS use case fails but it is linked to configuration, solutions are discussed on joid etherpad. onos scenario (run only 2 times on Orange POD 2) fails due to vPing/ssh ^ Scenario ^ VIM |||^ Controllers ||^ Features ||^ Comments ^ | | vPing SSH | vPing userdata | Tempest | Rally | ODL | ONOS | OVNO | PROMISE | vIMS | SDNVPN | | nosdn-nofeature [1] | {{:weather-clear.png?32}} | {{:weather-clear.png?32}} | {{:weather-clear.png?32}} | {{:weather-clear.png?32}} | N.R | N.R | N.R | {{:weather-clear.png?32}} | {{:weather-storm.png?32}} | N.R | criteria reached on the 21 ( vIMS excluded). Tempest rate 88% but some errors can be explained by config so can be removed from the equation leading to 93% | | odl_l2-ha | {{:weather-clear.png?32}} | {{:weather-clear.png?32}} | {{:weather-clear.png?32}} | {{:weather-clear.png?32}} | {{:weather-clear.png?32}} | N.R | N.R | {{:weather-clear.png?32}} | {{:weather-storm.png?32}} | N.R | Criteria reached on the 21 (vIMS excluded). Tempest rate = 98% | | onos-ha | {{:weather-storm.png?32}} | N.R. | {{:weather-overcast.png?32}} | {{:weather-few-clouds.png?32}} | N.R | {{:weather-few-clouds.png?32}} | N.R. | {{:weather-clear.png?32}} | N.R. | N.R | | | ocl-nofeature-ha | | | | | | | | | | | \\ IMPORTANT: intel-pod5 and intel-pod6 won't count for the results report until we fix the possible problems why we get those big timeouts. POD: intel-pod5 \\ source: https://build.opnfv.org/ci/view/joid/job/functest-joid-intel-pod5-daily-brahmaputra/ ^ Scenario ^ VIM |||^ Controllers ||^ Features ||^ Iteration ^ Comments ^ | | vPing SSH | vPing userdata | Tempest | Rally [1] | ODL | ONOS | OVNO | PROMISE | vIMS | SDNVPN | || | nosdn-nofeature | | | | | N.R | N.R | N.R | | | N.R | | | | odl_l2-ha | {{:weather-overcast.png?32}} | {{:weather-overcast.png?32}} | {{:weather-storm.png?32}} | {{:weather-overcast.png?32}} | {{:weather-few-clouds.png?32}} | N.R | N.R | {{:weather-few-clouds.png?32}} | {{:weather-storm.png?32}} | N.R | | | | onos-ha | | N.R | | | N.R | | N.R | | | N.R | | | | ocl-nofeature-ha | | | | | | | | | | | | | [1] Rally takes 4 hr to execute. Something is wrong here... \\ POD: intel-pod6 \\ source: https://build.opnfv.org/ci/view/functest/job/functest-joid-intel-pod6-daily-master/ ^ Scenario ^ VIM |||^ Controllers ||^ Features ||^ Iteration ^ Comments ^ | | vPing SSH | vPing userdata | Tempest | Rally | ODL | ONOS | OVNO | PROMISE | vIMS | SDNVPN | || | nosdn-nofeature [1] | | | | | N.R | N.R | N.R | | | N.R | | | | odl_l2-ha [1] | {{:weather-few-clouds.png?32}} | {{:weather-few-clouds.png?32}} | {{:weather-storm.png?32}} | {{:weather-few-clouds.png?32}} | {{:weather-few-clouds.png?32}} | N.R | N.R | {{:weather-overcast.png?32}} | {{:weather-storm.png?32}} | N.R | | 10 errors on promise test case out of 32 | | onos-ha | {{:weather-storm.png?32}} [2] | N.R. | {{:weather-storm.png?32}} | {{:weather-overcast.png?32}} | N.R | {{:weather-few-clouds.png?32}} | N.R. | {{:weather-overcast.png?32}} | {{:weather-storm.png?32}} | N.R | | 10 errors on promise test case out of 32 | | ocl-nofeature-ha | | | | | | | | | | | | | [1] No runs found with this scenario on this pod [2]: It seems the instances don't get an IP from the neutron dhcp agent... \\ \\ ==== Brahmaputra Troubleshooting ==== * [[vPing_Brahmaputra_page|vPing]] * [[odl_Brahmaputra_page|odl]] * [[onos_brahmaputra_page|onos]] * [[tempest_Brahmaputra_page|tempest]] * [[vIms_Brahmaputra_page|vIMS]] * [[rally_Brahmaputra_page | Rally]]