User Tools

Site Tools


This is an old revision of the document!

Base principles of the testing

Key objectives:

  • Define tooling for tests
  • Define test suites (SLA)
  • Installation and configuration of the tools
  • Automate test with CI

Procedure shall be independant from the installer. The procedure shall be performed on any of the installer.

Test tools

TODO: shall we be more prescritive on the toloing environment (creation of the VM, installation of the tools)?

Test scenarios


for release 1 we target the automation of the following tests

  • Rally scenarios including tempest
  • vPing (boot 2 VM, VM1 ping VM2, delete VM)
  • vIMS (Sipp VM trigering basic calls (REGISTER, INVITE) to clearwater compact vIMS VM)
  • ODL scenario?


The default scenario are:

  • authenticate
  • nova
  • cinder
  • glance
  • keystone
  • neutron
  • quotas
  • requests
  • tempest-do-not-run-against-production
  • heat
  • mistral
  • sahara
  • vm
  • ceilometer
  • designate
  • dummy
  • zaqar

the first ones (authenticate, nova, cinder, glance, keystone, neutron, quotas, requests, tempest-do-not-run-against-production) can be re-used. However scenario shall be tuned especially for the bench

  • which image (size, OS,..) TODO: check for reco in Spirent ETSI NFV doc on test
  • which SLA (booting time, error rate,..)

Note: during first manual launched on alpha Orange platform installed with opensteak installer, there were lots of errors (196) running Tempest scenario and some in Rally scenario (results to be analyzed)

Studies on the testcase shall be done

Open question

  • Shall we create new scenario (to remove tests or add new ones)?
  • not sure to be able to play the default scenario (ping VM) ⇒ which tool?

Test automation

To be discussed with CI team asap

status regarding installers

Jira ref Documentation Manual test Result analysis Automated test BGS link Comments
Rally Bench installation procedure described OK Tested with opensteack #3; Rally natively integrated in Fuell #1
Rally Tempest OK Tested with opensteack #3; Rally natively integrated in Fuell #1
vIMS Based on Clearwater solution
pharos_testing.1426752833.txt.gz · Last modified: 2015/03/19 08:13 by Morgan Richomme