This is an old revision of the document!
At a glance:
Date | Config | Tempest | vPing | Rally Bench | ODL |
---|---|---|---|---|---|
22/05 | O/S HA no ODL | 100 Tests / 26 Failures* | OK | some errors on Neutron and Nova benchs | N.A |
03/06 | target (O/S non HA + ODL) | 106 Tests / 14 Failures | OK | some errors on Neutron and Nova benchs | NOK |
04/06 | target | 96 /13 | OK | some erros in Neutron and Nova benchs | NOK |
Date | Config | Tempest | vPing | Rally Bench | ODL |
---|---|---|---|---|---|
22/05 | target (O/S HA + ODL) | 105 Tests / 32 Failures* | NOK | errors in Cinder, glance, Neutron, Nova benchs | N.A |
28/05 | target | 92-104 tests with 41-34 failures * | NOK either Boot Error or Timeout (both VMs properly booted but no ping seen in nova console | errors in Cinder, Nova and Neutron benchs | 18 tests / 12 failures |
03/06 | target | 98 tests with 37 failures* | OK | errors in Cinder, Nova and Neutron benchs | 18 tests / 3 failures ** |
04/06 | target | 98/37 | OK | erros in Cinder, Nova & Neutron benchs | 18/3 |
*: stable - 10 "false" failures identified linked to keystone identity v3 ⇒ admin_domain_name=default must be precised in tempest.conf
* *: Due to a known ODL bug: https://bugs.opendaylight.org/show_bug.cgi?id=3052
for release 1 we target the automation of the following tests
At the end of a Fresh install, the status of the OPNFV solutions according to the selected installer can be summarized as follow:
Fuel | Foreman | |
---|---|---|
Images | TestVM (Cirros) | none |
Networks | public: net04_ext+router, private: net04 | public: provider-network + router |
Flavors | m1.tiny, m1.small, m1.medium, m1.large, m1.xlarge, m1.micro | m1.small, m1.medium, m1.large, m1.xlarge |
OpenStack creds | OS_USERNAME=admin OS_PASSWORD=admin, OS_TENANT_NAME=admin | OS_USERNAME=admin OS_PASSWORD=octopus OS_TENANT_NAME=admin |
See reference doc
See Octopus etherpad: https://etherpad.opnfv.org/p/octopus
Community platforms connected to CI
For SR1, we use only one POD (LF POD2). This POD is used to test 2 version (Arno stable, Master(SR1) on the 2 installers (Fuel, Foreman)
Version | CI | vPing | ODL | Rally | Tempest | Comments |
---|---|---|---|---|---|---|
Arno stable foreman | OK | OK (~ 130s) | failed | OK | OK (107 test, 25 failed) | |
Arno stable fuel | not run (deploy failed) | used to run with similar results than master | ||||
Arno master foreman | OK | OK (~130s) | NOK | OK | OK (113 tests, 30 failed) | |
Arno master fuel | OK | OK (~80s) | idem Arno master foreman | OK | OK (113 tests, 13 failed) |
Version | CI | vPing | ODL | Rally | Tempest | Comments |
---|---|---|---|---|---|---|
Fuel SR0 | OK | OK (~ 80s) http://tinyurl.com/oucukrq | ODL container no integrated in installation | OK | OK (96 tests, 13 failed) http://tinyurl.com/oha7cr8 | |
Fuel SR1 #39 | OK | OK (~ 22s !!!) (average on last successful tests: 43s ) | 18 tests total, 15 passed, 3 failed | OK | OK (106 tests,16 failed) | |
Foreman SR0 | OK | OK (~100s) http://tinyurl.com/oucukrq | 18 tests/3 errors. Due to a known ODL bug: http://tinyurl.com/pasevk9 | OK | OK (98 tests, 37 failed) http://tinyurl.com/oha7cr8 | |
Foreman SR1 #17 | OK | OK (105s) (average on last 5 succesful tests: 110s) | 18 tests total, 15 passed, 3 failed | OK | OK (107 tests, 29 failed) |