User Tools

Site Tools


This is an old revision of the document!

Base principles of the testing

Key objectives:

  • Define tooling for tests
  • Define test suites (SLA)
  • Installation and configuration of the tools
  • Automate test with CI

Pad dedicated to func test:

IRC chan: #opnfv-testperf

Procedure shall be independant from the installer. The procedure shall be performed on any of the installer.

Test tools

Intel POD2 (contact Trevor Cooper) is dedicated to functional testing.

Functional tests shall be

  • independant from the installer (Fuel, Foreman/Puppet, Juju,..)
  • automated and integrated in CI

TODO: shall we be more prescritive on the toloing environment (creation of the VM, installation of the tools)?

Test scenarios


for release 1 we target the automation of the following tests

  • Rally scenarios including tempest
  • vPing (boot 2 VM, VM1 ping VM2, delete VM)
  • vIMS (Sipp VM trigering basic calls (REGISTER, INVITE) to clearwater compact vIMS VM)
  • ODL scenario?

List of testcases can be found here


The default scenario are:

  • authenticate
  • nova
  • cinder
  • glance
  • keystone
  • neutron
  • quotas
  • requests
  • tempest-do-not-run-against-production
  • heat
  • mistral
  • sahara
  • vm
  • ceilometer
  • designate
  • dummy
  • zaqar

the first ones (authenticate, nova, cinder, glance, keystone, neutron, quotas, requests, tempest-do-not-run-against-production) can be re-used. However scenario shall be tuned especially for the bench

  • which image (size, OS,..) TODO: check for reco in Spirent ETSI NFV doc on test
  • which SLA (booting time, error rate,..)

Note: during first manual launched on alpha Orange platform installed with opensteak installer, there were lots of errors (196) running Tempest scenario and some in Rally scenario (results to be analyzed)

Studies on the testcase shall be done

Open question

  • Shall we create new scenario (to remove tests or add new ones)?
  • not sure to be able to play the default scenario (ping VM) ⇒ which tool?

Test automation

Octopus etherpad:

Community platforms connected to CI

  • Ericsson
  • Intel (in progress)
  • Huawei
  • Orange (TBDone)

status regarding installers

Jira ref Documentation Manual test Result analysis Automated test BGS link Comments
Rally Bench installation procedure described OK Installed on Jump host server of Intel POD 2 #1; Rally natively integrated in Fuell #2;Tested with opensteack #3; Morgan. Tempest not working on POD - same issue than on #3, patch applied but seems there is a pb ⇒ contact openstack-rally. Rally Test suite can be performed but flavours, images are missing on Openstack SUT deployed on POD 2
Tempest OK Use of khalisi for foreman/puppet #1; Rally natively integrated in Fuell #2;Tested with opensteack #3;
vPing Sama
vIMS Based on Clearwater solution Martin
ODL Peter ⇒
pharos_testing.1428047059.txt.gz · Last modified: 2015/04/03 07:44 by Morgan Richomme