pharos_testing
Differences
This shows you the differences between two versions of the page.
| Both sides previous revision Previous revision Next revision | Previous revision | ||
|
pharos_testing [2015/03/19 07:42] Morgan Richomme [status regarding installers] |
pharos_testing [2015/11/05 22:16] (current) Trevor Cooper |
||
|---|---|---|---|
| Line 1: | Line 1: | ||
| - | ====== Base principles of the testing ====== | + | Pharos is dedicated to test infrastructure (specification, process, ...) |
| - | Key objectives: | + | Functional tests can be found on functest project page: [[opnfv_functional_testing|Project: OPNFV - Base system functionality testing]] |
| - | * Define tooling for tests | + | |
| - | * Define test suites (SLA) | + | |
| - | * Installation and configuration of the tools | + | |
| - | * Automate test with CI | + | |
| - | Procedure shall be independant from the installer. | + | Test strategy from Arno to B-Release: {{:test_strategy.pptx|}} |
| - | The procedure shall be performed on any of the installer. | + | |
| - | + | ||
| - | ===== Test tools ===== | + | |
| - | + | ||
| - | * [[https://wiki.openstack.org/wiki/Rally|Rally]] (benchmark, Tempest) | + | |
| - | * [[http://robotframework.org/|Robot framework]] | + | |
| - | * [[http://sipp.sourceforge.net/|Sipp]] (for SIP related test e.g vIMS) | + | |
| - | + | ||
| - | TODO: shall we be more prescritive on the toloing environment (creation of the VM, installation of the tools)? | + | |
| - | + | ||
| - | + | ||
| - | ===== Test scenarios ===== | + | |
| - | + | ||
| - | === Overview === | + | |
| - | + | ||
| - | for release 1 we target the automation of the following tests | + | |
| - | * Rally scenarios including tempest | + | |
| - | * vPing (boot 2 VM, VM1 ping VM2, delete VM) | + | |
| - | * vIMS (Sipp VM trigering basic calls (REGISTER, INVITE) to clearwater compact vIMS VM) | + | |
| - | * ODL scenario? | + | |
| - | + | ||
| - | === Rally === | + | |
| - | + | ||
| - | The default scenario are: | + | |
| - | * authenticate | + | |
| - | * nova | + | |
| - | * cinder | + | |
| - | * glance | + | |
| - | * keystone | + | |
| - | * neutron | + | |
| - | * quotas | + | |
| - | * requests | + | |
| - | * tempest-do-not-run-against-production | + | |
| - | * heat | + | |
| - | * mistral | + | |
| - | * sahara | + | |
| - | * vm | + | |
| - | * ceilometer | + | |
| - | * designate | + | |
| - | * dummy | + | |
| - | * zaqar | + | |
| - | + | ||
| - | the first ones (authenticate, nova, cinder, glance, keystone, neutron, quotas, requests, tempest-do-not-run-against-production) can be re-used. | + | |
| - | However scenario shall be tuned especially for the bench | + | |
| - | * which image (size, OS,..) TODO: check for reco in Spirent ETSI NFV doc on test | + | |
| - | * which SLA (booting time, error rate,..) | + | |
| - | + | ||
| - | Note: during first manual launched on alpha Orange platform installed with opensteak installer, there were lots of errors (196) running Tempest scenario and some in Rally scenario (results to be analyzed) | + | |
| - | + | ||
| - | Studies on the testcase shall be done | + | |
| - | + | ||
| - | === Open question === | + | |
| - | * Shall we create new scenario (to remove tests or add new ones)? | + | |
| - | * not sure to be able to play the default scenario (ping VM) => which tool? | + | |
| - | + | ||
| - | + | ||
| - | + | ||
| - | ===== Test automation ===== | + | |
| - | + | ||
| - | To be discussed with CI team asap | + | |
| - | + | ||
| - | + | ||
| - | ===== status regarding installers ===== | + | |
| - | + | ||
| - | + | ||
| - | ^ "Func test" ^ "Jira" ^ "Description/Documentation" ^ "Manual test" ^ "Result analysis" ^ "Automated test" ^ "BGS link" ^ "Comments" ^ | + | |
| - | |Rally Bench| |installation procedure described [[https://github.com/Orange-OpenSource/opnfv/blob/master/docs/TEST.md]]|OK| | |tested on Orange alpha testbed, Rally natively integrated in Fuel| | | + | |
| - | |Rally Tempest| | | | | | | | | + | |
| - | |vPing| | | | | | | | | + | |
| - | |vIMS| |based on clearwater solution| | | | | | | + | |
| - | |ODL| | | | | | | | | + | |
| - | + | ||
| - | + | ||
| - | + | ||
| - | + | ||
| - | + | ||
| - | + | ||
| - | + | ||
pharos_testing.1426750949.txt.gz · Last modified: 2015/03/19 07:42 by Morgan Richomme
Page Tools
Except where otherwise noted, content on this wiki is licensed under the following license: Creative Commons Attribution 4.0 International License
