User Tools

Site Tools



This shows you the differences between two versions of the page.

Link to this comparison view

Both sides previous revision Previous revision
Next revision
Previous revision
Next revision Both sides next revision
opnfv_functional_testing [2015/10/06 12:29]
Morgan Richomme
opnfv_functional_testing [2015/12/09 14:51]
Morgan Richomme
Line 92: Line 92:
 ===== Tips ===== ===== Tips =====
 +==== Short story on Functest versus the other test projects and the installers ====
 +First of all we try to be as agnostic as possible.
 +we see OPNFV as a black box.
 +We automatically run functest-{installer}-master after a successful fresh installation of {installer}
 +In functest (and in yardstick) we have internal test cases and companion test projects
 +the functest companion projects dealing with functional testing are
 +  * doctor
 +  * copper
 +  * ovno
 +  * policyTest
 +  * promise
 +  * sdnvpn
 +companion project means that we wil onboard them in Functest CI process. After a fresh install, functest will be started (a docker file) running several tests one after the others (e.g. https://​​ci/​view/​functest/​job/​functest-fuel-master/​lastSuccessfulBuild/​consoleText) including companion tests in the near future.
 +each test may need specific env, so functest can install tooling and triggers scripts to setup env
 +we will also create a filtering mechanism to run tests only when possible (assuming that by default it is always possible) but no need to run odl suite towards an OPNFV including onos
 +In functest we developed ​ the code for our internal tests (scripts, tests, integration of upstream test suites,...)
 +The companion projects are in charge of their scripts, their code, their scenario...
 +we mutualize the CI env (running on target CI production labs), the tooling (some projects need for instance to run upstream ODL suite, functest install once robot and mininet for all of them, Rally may test module used by feature project (eg; congress/​copper)),​ the reporting, the documentation
 +Then jenkins functest-{installer} job will produce results that will be pushed in the Test collection DB
 +before Release B we will look carefully to the jobs (success criteria for Functest for B-Release is 4 successfull run on CI of the onboarded projects)
 +  * if errors in integration or internal cases => we will blame ourselves
 +  * if erros in companion projects => we will report the issues to the member of the feature projects
 +So you can see that we did not do specific stuff for 1 installer.
 +We have some adaptations of course in our scripts (e.g. to retrieve the credentials - not the same way on the different installers) but we do not create a plugin for a given installers, we have a docker file.
 +Functest (as yardstick) docker file could even be almost used towards any OPNFV-like solution (some adaptations on the env setup) ​
 ==== Git ==== ==== Git ====
Line 122: Line 157:
 Want to Join: [[https://​​ns=%3A&​image=%3Aopnfv-_functest.pdf&​do=media&​tab_files=files&​tab_details=view| know more on Functest]] Want to Join: [[https://​​ns=%3A&​image=%3Aopnfv-_functest.pdf&​do=media&​tab_files=files&​tab_details=view| know more on Functest]]
 +{{:​devtest_journey.pdf|Tester Journey}}
opnfv_functional_testing.txt ยท Last modified: 2016/03/01 08:48 by Ryota Mibu