functextnexttaks
Differences
This shows you the differences between two versions of the page.
| Both sides previous revision Previous revision Next revision | Previous revision | ||
|
functextnexttaks [2016/02/22 08:09] Morgan Richomme [Table] |
functextnexttaks [2016/03/15 09:48] (current) Morgan Richomme |
||
|---|---|---|---|
| Line 1: | Line 1: | ||
| - | ====== Functest R2 Brahmaputra page ====== | + | == Colorado release == |
| - | [[https://etherpad.opnfv.org/p/functiontest|etherpad to discuss]] | ||
| - | ==== Release Info ==== | + | {{:first_presentation.pptx| introduction for Flash test}} |
| - | + | ||
| - | + | ||
| - | === Summary === | + | |
| - | + | ||
| - | ^ Criteria ^ Functest Status ^ | + | |
| - | | 1. Project name | Functest | | + | |
| - | | 2. Project status indicator for milestone C readiness: Green/Yellow/Red; GREEN = on track; YELLOW = working minor issues to get back on track; RED = offtrack with major issues | Yellow/Red | | + | |
| - | | 3.Name of person providing status | Morgan Richomme, Jose Lausuch | | + | |
| - | | 4.a link to your Jira tasks (filtered on R2) | [[https://jira.opnfv.org/issues/?jql=project%20%3D%20FUNCTEST%20%26%20labels%20%3D%20R2| Functest R2 JIRA]] | | + | |
| - | | 5.a link to your Jira dashboard | [[https://jira.opnfv.org/secure/Dashboard.jspa?selectPageId=10611|JIRA Dashboard]] [[https://jira.opnfv.org/secure/RapidBoard.jspa?rapidView=59|Kanban]] | | + | |
| - | + | ||
| - | ==== Brahmaputra Summary ==== | + | |
| - | + | ||
| - | Please note that you may have a dynamic view per [[http://testresults.opnfv.org/proto/index2.html|installer]], [[http://testresults.opnfv.org/proto/index-pod.html|pod]] or [[http://testresults.opnfv.org/proto/index-scenario.html|scenario]] | + | |
| - | + | ||
| - | New Functest [[http://testresults.opnfv.org/dashboard/|dashboard]] look. | + | |
| - | + | ||
| - | this table reflects the status but it is not dynamically up to date as CI never stops.. | + | |
| - | + | ||
| - | We consider hereafter only baremetal scenario. | + | |
| - | + | ||
| - | + | ||
| - | Icon description: | + | |
| - | + | ||
| - | {{:weather-clear.png?32}} : 4+ consecutive runs passing the success criteria\\ | + | |
| - | {{:weather-few-clouds.png?32}} : <4 successful consecutive runs but passing the success criteria\\ | + | |
| - | {{:weather-overcast.png?32}} : close to pass the success criteria\\ | + | |
| - | {{:weather-storm.png?32}} : 0% success, not passing\\ | + | |
| - | + | ||
| - | + | ||
| - | === Fuel === | + | |
| - | POD: ericsson-pod2 \\ | + | |
| - | source: https://build.opnfv.org/ci/view/functest/job/functest-fuel-ericsson-pod2-daily-brahmaputra/ | + | |
| - | ^ Scenario ^ VIM |||^ Controllers ||^ Features ||^ Iteration ^ Comments ^ | + | |
| - | | | vPing SSH | vPing userdata | Tempest [1] | Rally | ODL | ONOS | OVNO | PROMISE | vIMS | SDNVPN | || | + | |
| - | | nosdn-nofeature | {{:weather-clear.png?32}} | {{:weather-clear.png?32}} | {{:weather-overcast.png?32}} | {{:weather-clear.png?32}} | N.R | N.R | N.R | {{:weather-clear.png?32}} | {{:weather-clear.png?32}} | N.R | | Tempest success rate 86% | | + | |
| - | | odl_l3-ha | {{:weather-few-clouds.png?32}} [3] | {{:weather-clear.png?32}} | {{:weather-overcast.png?32}} | {{:weather-clear.png?32}} | {{:weather-clear.png?32}} | N.R | N.R | {{:weather-clear.png?32}} | {{:weather-clear.png?32}} | N.R | | Tempest success rate 87% | | + | |
| - | | odl_l2-ha | {{:weather-clear.png?32}} | {{:weather-clear.png?32}} | {{:weather-overcast.png?32}} | {{:weather-clear.png?32}} | {{:weather-clear.png?32}} | N.R | N.R | {{:weather-clear.png?32}} | {{:weather-clear.png?32}} | N.R | | Tempest success rate 83% | | + | |
| - | | odl_l2-bgpvpn-noha | | | | | | | | | | | | | | + | |
| - | | onos-ha | {{:weather-few-clouds.png?32}} [2] | N.R | {{:weather-overcast.png?32}} | {{:weather-clear.png?32}} | N.R | {{:weather-clear.png?32}} | N.R | {{:weather-clear.png?32}} | N.R | N.R | | Tempest success rate 87% | | + | |
| - | | nosdn-ovs-ha | {{:weather-clear.png?32}} | {{:weather-clear.png?32}} | {{:weather-overcast.png?32}} | {{:weather-clear.png?32}} | N.R | N.R | N.R | {{:weather-clear.png?32}} | {{:weather-clear.png?32}} | N.R | | Tempest success rate 87% | | + | |
| - | | nosdn-kvm-ha | | | | | | | | | | | | | | + | |
| - | | nosdn-ovs_kvm-ha | | | | | | | | | | | | | | + | |
| - | + | ||
| - | + | ||
| - | + | ||
| - | [1]: Detected some errors (~7-8) possibly due to insufficient IPs in the public network ip range. Investigation in progress. Possible fix by Jonas: https://gerrit.opnfv.org/gerrit/#/c/10159/ | + | |
| - | + | ||
| - | [2]: It failed 2 times in the last 5 runs.. waiting for more runs to see if it is impracticable. Ashley proposed a patch to fix this and yardstick problems with SSH in this scenario | + | |
| - | + | ||
| - | [3]: last 2 runs it has failed | + | |
| - | + | ||
| - | \\ | + | |
| - | \\ | + | |
| - | + | ||
| - | + | ||
| - | + | ||
| - | === Apex === | + | |
| - | POD: opnfv-jump-1\\ | + | |
| - | source: https://build.opnfv.org/ci/view/functest/job/functest-apex-opnfv-jump-1-daily-brahmaputra/ \\ | + | |
| - | + | ||
| - | ^ Scenario ^ VIM |||^ Controllers ||^ Features ||^ Iteration ^ Comments ^ | + | |
| - | | | vPing SSH | vPing userdata | Tempest | Rally | ODL | ONOS | OVNO | PROMISE | vIMS | SDNVPN | || | + | |
| - | | odl_l3-ha | {{:weather-storm.png?32}} | {{:weather-storm.png?32}} | {{:weather-overcast.png?32}} | {{:weather-clear.png?32}} | {{:weather-clear.png?32}} | N.R | N.R | Not executed yet | {{:weather-storm.png?32}} | N.R | | Tempest success rate 70% | | + | |
| - | | odl_l2-ha | {{:weather-clear.png?32}} | {{:weather-clear.png?32}} | {{:weather-overcast.png?32}} | {{:weather-clear.png?32}} | {{:weather-clear.png?32}} | N.R | N.R | Not executed yet | {{:weather-clear.png?32}} | N.R. | | Tempest success rate 82% | | + | |
| - | | onos-ha | {{:weather-storm.png?32}} | N.R [1] | {{:weather-overcast.png?32}} | {{:weather-clear.png?32}} | N.R | {{:weather-clear.png?32}} | N.R | Not executed yet | N.R. | N.R. | | Tempest success rate 87 % | | + | |
| - | | odl_l2-sfc | | | | | | | | | | | | | | + | |
| - | | ocl-nofeature-ha | | | | | | | | | | | | | | + | |
| - | + | ||
| - | [1] Nova metadata service is not working properly with ONOS yet. Maybe in SR1. | + | |
| - | + | ||
| - | + | ||
| - | + | ||
| - | \\ | + | |
| - | \\ | + | |
| - | + | ||
| - | + | ||
| - | + | ||
| - | === Compass === | + | |
| - | POD: huawei-us-deploy-bare-1 \\ | + | |
| - | source: https://build.opnfv.org/ci/view/functest/job/functest-compass-huawei-us-deploy-bare-1-daily-brahmaputra/ | + | |
| - | ^ Scenario ^ VIM |||^ Controllers ||^ Features ||^ Iteration ^ Comments ^ | + | |
| - | | | vPing SSH | vPing userdata | Tempest | Rally | ODL | ONOS | OVNO | PROMISE | vIMS | SDNVPN | || | + | |
| - | | nosdn-nofeature | {{:weather-clear.png?32}} | {{:weather-clear.png?32}} | {{:weather-clear.png?32}} | {{:weather-clear.png?32}} | N.R | N.R | N.R | N.R | {{:weather-clear.png?32}} | N.R | 4 | success criteria reached 8/2 | | + | |
| - | | odl_l2-ha | {{:weather-clear.png?32}} | {{:weather-clear.png?32}} | {{:weather-clear.png?32}} | {{:weather-clear.png?32}} | {{:weather-clear.png?32}} | N.R | N.R | N.R | {{:weather-clear.png?32}} | N.R | 4 | success criteria reached 9/2 | | + | |
| - | | onos-ha | {{:weather-clear.png?32}} | N.R | {{:weather-clear.png?32}} [1] | {{:weather-clear.png?32}} | N.R | {{:weather-clear.png?32}} | N.R. | N.R | N.R | N.R | 4 | success criteria reached 4/2 | | + | |
| - | | ocl-nofeature-ha | | | | | | | | | | | | | | + | |
| - | + | ||
| - | [1]: There are 11 failed tests out of 210 which (we think) are related to nova metadata service not supported by onos yet. Some tests need to pass the ssh-key to the instances using that service. However, it passes the success criteria of >=90% | + | |
| - | + | ||
| - | + | ||
| - | + | ||
| - | + | ||
| - | \\ | + | |
| - | \\ | + | |
| - | + | ||
| - | === Joid === | + | |
| - | POD: orange-pod2 \\ | + | |
| - | source: https://build.opnfv.org/ci/view/functest/job/functest-joid-orange-pod2-daily-master/ | + | |
| - | ^ Scenario ^ VIM |||^ Controllers ||^ Features ||^ Iteration ^ Comments ^ | + | |
| - | | | vPing SSH | vPing userdata | Tempest | Rally | ODL | ONOS | OVNO | PROMISE | vIMS | SDNVPN | || | + | |
| - | | nosdn-nofeature [1] | {{:weather-few-clouds.png?32}} | {{:weather-few-clouds.png?32}} | {{:weather-overcast.png?32}} | {{:weather-few-clouds.png?32}} | N.R | N.R | N.R | {{:weather-few-clouds.png?32}} | {{:weather-storm.png?32}} | N.R | | Tempest 26 errors/210 87% | | + | |
| - | | odl_l2-ha | {{:weather-clear.png?32}} | {{:weather-clear.png?32}} | {{:weather-few-clouds.png?32}} | {{:weather-clear.png?32}} | {{:weather-clear.png?32}} | N.R | N.R | {{:weather-clear.png?32}} | {{:weather-storm.png?32}} | N.R | | Tempest: 17% \\ : > 90% | | + | |
| - | | onos-ha | {{:weather-storm.png?32}} | N.R. | {{:weather-overcast.png?32}} | {{:weather-few-clouds.png?32}} | N.R | {{:weather-few-clouds.png?32}} | N.R. | {{:weather-few-clouds.png?32}} | N.R. | N.R | | | | + | |
| - | | ocl-nofeature-ha | | | | | | | | | | | | | | + | |
| - | + | ||
| - | + | ||
| - | + | ||
| - | + | ||
| - | \\ | + | |
| - | + | ||
| - | IMPORTANT: intel-pod5 and intel-pod6 won't count for the results report until we fix the possible problems why we get those big timeouts. | + | |
| - | + | ||
| - | POD: intel-pod5 \\ | + | |
| - | source: https://build.opnfv.org/ci/view/joid/job/functest-joid-intel-pod5-daily-brahmaputra/ | + | |
| - | ^ Scenario ^ VIM |||^ Controllers ||^ Features ||^ Iteration ^ Comments ^ | + | |
| - | | | vPing SSH | vPing userdata | Tempest | Rally [1] | ODL | ONOS | OVNO | PROMISE | vIMS | SDNVPN | || | + | |
| - | | nosdn-nofeature | | | | | N.R | N.R | N.R | | | N.R | | | | + | |
| - | | odl_l2-ha | {{:weather-overcast.png?32}} | {{:weather-overcast.png?32}} | {{:weather-storm.png?32}} | {{:weather-overcast.png?32}} | {{:weather-few-clouds.png?32}} | N.R | N.R | {{:weather-few-clouds.png?32}} | {{:weather-storm.png?32}} | N.R | | | | + | |
| - | | onos-ha | | N.R | | | N.R | | N.R | | | N.R | | | | + | |
| - | | ocl-nofeature-ha | | | | | | | | | | | | | | + | |
| - | + | ||
| - | + | ||
| - | + | ||
| - | + | ||
| - | [1] Rally takes 4 hr to execute. Something is wrong here... | + | |
| - | + | ||
| - | + | ||
| - | \\ | + | |
| - | + | ||
| - | POD: intel-pod6 \\ | + | |
| - | source: https://build.opnfv.org/ci/view/functest/job/functest-joid-intel-pod6-daily-master/ | + | |
| - | ^ Scenario ^ VIM |||^ Controllers ||^ Features ||^ Iteration ^ Comments ^ | + | |
| - | | | vPing SSH | vPing userdata | Tempest | Rally | ODL | ONOS | OVNO | PROMISE | vIMS | SDNVPN | || | + | |
| - | | nosdn-nofeature [1] | | | | | N.R | N.R | N.R | | | N.R | | | | + | |
| - | | odl_l2-ha [1] | {{:weather-few-clouds.png?32}} | {{:weather-few-clouds.png?32}} | {{:weather-storm.png?32}} | {{:weather-few-clouds.png?32}} | {{:weather-few-clouds.png?32}} | N.R | N.R | {{:weather-overcast.png?32}} | {{:weather-storm.png?32}} | N.R | | 10 errors on promise test case out of 32 | | + | |
| - | | onos-ha | {{:weather-storm.png?32}} [2] | N.R. | {{:weather-storm.png?32}} | {{:weather-overcast.png?32}} | N.R | {{:weather-few-clouds.png?32}} | N.R. | {{:weather-overcast.png?32}} | {{:weather-storm.png?32}} | N.R | | 10 errors on promise test case out of 32 | | + | |
| - | | ocl-nofeature-ha | | | | | | | | | | | | | | + | |
| - | + | ||
| - | + | ||
| - | [1] No runs found with this scenario on this pod | + | |
| - | + | ||
| - | [2]: It seems the instances don't get an IP from the neutron dhcp agent... | + | |
| - | + | ||
| - | + | ||
| - | \\ | + | |
| - | \\ | + | |
| - | + | ||
| - | ==== Brahmaputra Troubleshooting ==== | + | |
| - | + | ||
| - | * [[vPing_Brahmaputra_page|vPing]] | + | |
| - | * [[odl_Brahmaputra_page|odl]] | + | |
| - | * [[onos_brahmaputra_page|onos]] | + | |
| - | * [[tempest_Brahmaputra_page|tempest]] | + | |
| - | * [[vIms_Brahmaputra_page|vIMS]] | + | |
| - | * [[rally_Brahmaputra_page | Rally]] | + | |
| - | + | ||
| - | ==== Associated Functest Wiki pages ==== | + | |
| - | + | ||
| - | * [[https://wiki.opnfv.org/opnfv_test_dashboard | test dashboard]] | + | |
| - | * [[https://wiki.opnfv.org/collection_of_test_results| test result collection API]] | + | |
| - | * [[https://wiki.opnfv.org/traffic_profile_management | traffic profile management ]] | + | |
| - | * [[https://wiki.opnfv.org/onos_test_suite | ONOS suite]] | + | |
| - | * [[https://wiki.opnfv.org/functest_docker| functest dockerization]] | + | |
| - | * [[https://wiki.opnfv.org/feature_test_project_matrix| Cross-projects dependencies]] | + | |
| - | * [[ODL new test suites]] | + | |
| - | + | ||
| - | + | ||
| - | ==== Feature projects dependencies ==== | + | |
| - | + | ||
| - | ^ Feature project ^ actions for functest ^ Contacts ^ | + | |
| - | | Copper | if integrated in B-release, goal will consist in adding tempest/Rally tests related to Congress | Brian | | + | |
| - | | Doctor | test fully managed by Doctor team. Goal for functest consists in integrating the tests into the suite and run it on the different target CI labs | Ryota | | + | |
| - | | ONOSFW | create a test suite | Managed by Qinglong | | + | |
| - | | OVNO | see [[http://ircbot.wl.linuxfoundation.org/meetings/opnfv-testperf/2015/opnfv-testperf.2015-11-24-16.46.html|minutes]] meeting with Stuart | Stuart | | + | |
| - | | PolicyTest | extend ODL test suites with GBP tests already created upstream in ODL | Kieth / Peter & Amaged | | + | |
| - | | Promise | wait for feedback | peter Lee | | + | |
| - | | SDNVPN | RF tests for ODL VPN service using ODL REST API (import from ODL) | Tim Irnich / José | | + | |
| - | + | ||
| - | + | ||
| - | SFC managed by yardstick | + | |
| - | + | ||
| - | + | ||
| - | ==== Studies ==== | + | |
| - | + | ||
| - | === VNF === | + | |
| - | + | ||
| - | * vPE: [[https://wiki.opnfv.org/virtual_provider_edge#generic_functional_testing|vPE Function Test Suites]] | + | |
| - | * vEPC: ? | + | |
| - | * vCDN: ? | + | |
| - | * vSBC | + | |
| - | * vIMS extension (distributed solution, RTP,..) | + | |
| - | * vXX | + | |
| - | * ... | + | |
| - | + | ||
| - | === New scenarios === | + | |
| - | + | ||
| - | For Arno, we reused existing upstream suites (except vIMS). | + | |
| - | For Rally bench scenario, we aggregated the test in 1 scenario per module but we did not create original scenarios. It is time to think to our own scenario... | + | |
| - | + | ||
| - | === Exploitation of existing tests === | + | |
| - | + | ||
| - | Rally bench tests will provide figures (booting time, authentication,..). These tests will be performed on several testbed with the same OPNFV solution and the same tooling. The difference will thus be linked to the hardware and/or network configuration. It will be interesting to collect all the results (available under ./results/opnfv-[module]-[date].json under each platform) and try to interprate them. | + | |
| - | + | ||
| - | + | ||
| - | === New tools === | + | |
| - | + | ||
| - | We shall consider the integration of new tools such as [[yardstick]|yardstick] | + | |
| - | + | ||
| - | === Synchro with ETSI Test group === | + | |
| - | + | ||
| - | Synchronisation to be planned (e.g. pre deployement document discussed in Prague => how to create testcases to be integrated in continuous integration..) | + | |
| - | + | ||
| - | + | ||
| - | + | ||
| - | === Roadmap === | + | |
| - | + | ||
| - | imagined in July 2015 | + | |
| - | + | ||
| - | {{ :functest_b-release2.png?600 |}} | + | |
| - | + | ||
| - | {{ :functest-b-release3.png?600 |}} | + | |
functextnexttaks.1456128547.txt.gz · Last modified: 2016/02/22 08:09 by Morgan Richomme
Page Tools
Except where otherwise noted, content on this wiki is licensed under the following license: Creative Commons Attribution 4.0 International License
