====== Functest for Release 1 (Arno) ======


===== Arno stable 4/6/2015 =====

[[https://wiki.opnfv.org/releases/arno/releasereadiness|R1 follow up]]

At a glance:

==== POD1 ====

^ Date   ^ Config                     ^ Tempest                   ^ vPing  ^ Rally Bench                             ^ ODL  ^
| 22/05  | O/S HA no ODL              | 100 Tests / 26 Failures*  | OK     | some errors on Neutron and Nova benchs  | N.A  |
| 03/06  | target (O/S non HA + ODL)  | 106 Tests / 14 Failures   | OK     | some errors on Neutron and Nova benchs  | NOK   |
| 04/06 | target | 96 /13 | OK | some erros in Neutron and Nova benchs| NOK |





==== POD2 ====

^ Date   ^ Config                 ^ Tempest                             ^ vPing                                                                                        ^ Rally Bench                                     ^ ODL                       ^   
| 22/05  | target (O/S HA + ODL)  | 105 Tests / 32 Failures*            | NOK                                                                                           | errors in Cinder, glance, Neutron, Nova benchs  | N.A                       |   
| 28/05  | target                 | 92-104 tests with 41-34 failures *  | NOK either Boot Error or Timeout (both VMs properly booted but no ping seen in nova console  | errors in Cinder, Nova and Neutron benchs       | 18 tests / 12 failures    | 
| 03/06  | target                 | 98 tests with 37 failures*          | OK                                                                                           | errors in Cinder, Nova and Neutron benchs       | 18 tests / 3 failures **  |   
| 04/06 | target | 98/37 | OK | erros in Cinder, Nova & Neutron benchs | 18/3 |







*: stable - 10 "false" failures identified linked to keystone identity v3 => admin_domain_name=default must be precised in tempest.conf

* *: Due to a known ODL bug: https://bugs.opendaylight.org/show_bug.cgi?id=3052


==== Scope ====

for release 1 we target the automation of the following tests
  * bench Rally scenarios
  * tempest Rally
  * vPing (boot 2 VM, VM1 ping VM2, delete VM)
  * ODL scenario / Robot framework

==== SUT ====

At the end of a Fresh install, the status of the OPNFV solutions according to the selected installer can be summarized as follow:

^                  ^ Fuel                                                         ^ Foreman                                                     ^
| Images           | TestVM (Cirros)                                              | none                                                        |
| Networks         | public: net04_ext+router, private: net04                     | public: provider-network + router                           |
| Flavors          | m1.tiny, m1.small, m1.medium, m1.large, m1.xlarge, m1.micro  | m1.small, m1.medium, m1.large, m1.xlarge                    |
| OpenStack creds  | OS_USERNAME=admin OS_PASSWORD=admin, OS_TENANT_NAME=admin    | OS_USERNAME=admin OS_PASSWORD=octopus OS_TENANT_NAME=admin  |

==== Functest configuration script ====

See reference [[http://artifacts.opnfv.org/functest/docs/functest.html|doc]]
==== Testcases follow-up ====

  * [[R1 Tempest]]
  * [[R1 Rally bench]]
  * [[R1 ODL suite]]
  * [[R1 vPing]]
 
==== Test automation  ====

See Octopus etherpad: [[https://etherpad.opnfv.org/p/octopus]]

Community platforms connected to CI
  * Ericsson
  * Intel
  * Huawei
  * Orange


[[Arno SR1 result page]]