Complementary to this wiki, we'll use https://etherpad.opnfv.org/p/bgs to track activity.
The test environment integrates the following components and is expected to deploy to bare metal as well as to a virtual environment. All setup and install procedures will be automated (no manual setup/testing). The deploy/test cycle is intended to be run periodically by a job scheduler (Jenkins) and produce a pass/fail result.
Outline of the environment
See Target System State for a complete list of components on versions.
The following set of use-cases will guide the testing performed as part of Bootstrap/Get-started. The project does not intent to provide a comprehensive coverage of the particular use-cases and all associated testing scenarios. We just need a few VNFs that we can run on the environment Bootstrap/Get-started provides. Having a few example VNFs (rather than only one) is to ensure that test scenarios are kept generic wherever possible. Use-case specific test projects are expected to provide proper and full coverage for specific use-cases.
|Simple VNF: vRouter||Bring up one or multiple instances of a simple virtual router. OpenWRT will be used as virtual router.||Joseph Gasparakis|
|Simple VNF: vIDS||Bring up one or multiple instances of a service appliance. Snort/IDS will be used as service appliance.||Joseph Gasparakis|
|SFC||Service chaining using open source components: Firewall and IDS (OpenWRT as simple firewall, Snort as IDS)|
This section lists tasks and test cases which Bootstrap/Get-started aspires to perform. It is expected that the scenario tests fold into more specific test projects (e.g. for system level functionality testing, specific VNF use-case tests, performance tests etc.) - which are in the process of being defined.
|Environment setup||Setup base hardware environment (servers, network connectivity)||Linux Foundation|
|Server boot up||PXE boot the servers and prepare them with base OS to load OPNFV components.|
|Install OPNFV components|| Automatic install OpenStack control and compute nodes (with an HA setup). |
Automatic install OpenDaylight (with an HA setup).
Automatic install of test harness components such as Tempest, Rally.
|Components function test: OpenStack||Ensure OpenStack components are installed correctly and functioning. Leverage OpenStack Tempest and OpenStack Rally. Includes smoke, baremetal, compute, data_processing, identity, image, network, object_storage, orchestration, telemetry, and volume testing (current list of tempest scenario tests available). See example for the result of a tempest run.|
|Components function test: OpenDaylight/Networking||Ensure OpenDaylight components are installed correctly and functioning. Check whether relevant Karaf modules are loaded and functioning correctly.|
|Scenario testing: Basic system testing|| Basic system testing leverages Tempest Scenario testing with Openstack deployment configured to use Neutron/ML2 managed networking supplied by OpenDaylight.|
* Basic network ops: Test network connectivity to a VM via a tenant network, a public network, or both.
* Advanced server networking: Test VM connectivity after some advanced instance operations executed: start/stop instance, reboot instance, migrate instance, etc. Tests prioritised by relevance to NFV workloads.
* Install and removal of ODL Karaf features on running instance.
* Security groups: Verify cross tenant connectivity - basic as well as after resizing an instance etc.
|Scenario testing: High-availability testing|| HA testing leveraging Tempest Scenario testing - component and physical infrastructure failures.|
Test component HA/failure-recovery for those components that support it:
* Nova scheduler
* Nova conductor
* Cinder scheduler
* Neutron server
* Heat engine
Test hardware and system failures by emulating failure scenarios
* dead server
* dead switch
* dead port
* dead disk
* full disk
|Simple VNF creation||Create VM images for vRouter (from OpenWRT) and vIDS (from Snort) for testing purposes||Frank|
|Simple VNF hosting: base||Scenario test using Tempest to create and configure a simple VNF: Create 3 VMs: 2 hosts and one instance of a vRouter (OpenWRT) with basic configuration. Test whether vRouter is reachable (within its domain) from the host and can pass packets between the hosts.|
|Simple VNF hosting: Scale/Idempotence testing||Scenario test using Tempest to create/remove n-instances of a vRouter. (Test requires a simple VNFM which in this case will be provided by a script)|
|Simple VNF hosting: Sequencing||Scenario test using Tempest to investigate system behavior in case components are not triggered/loaded in the correct sequence (i.e. VNFM would behave wrongly)|
|Simple VNF hosting: Resource control||Scenario test using Tempest to deploy n instances of vRouter and check for allocated vCPUs, check vCPU pinning.|
|Simple VNF hosting: Fault management||Scenario test using Tempest to test detection of VNF failure (vRouter)|
|Simple VNF hosting: Service upgrade||Scenario test using Tempest to replace an existing VNF (vRouter) with a new revision|
|SFC: base||Scenario test using Tempest to create a service chain from a set of open-source VNFs: Firewall and IDS: OpenWRT and Snort. Basic functionality test of the service chain.|
|Eventing/Statistics||Scenario tests to test capturing events and statistics from different system components (VNFs, ODL, OpenStack)|
|Virtual forwarder testing|| Test scenario descriptions for virtual forwarders (e.g. OVS) deployed by the OPNFV platform. Forwarding performance testing between two or more hosts: |
* Different traffic profiles (large packets, small packets, IMIX - unicast, multicast) and network configurations (QoS, different tunnel types, etc.)
* bare metal deployment (virtual forwarder directly deployed on host); Single/multiple virtual forwarders on the same instance
* virtual environments (VM based deployment); Single/multiple virtual forwarders on the same instance
|Sample VM testing||Test scenarios as for the forwarder, but using a VNF VM between source and destination to get realistic idea of performance in a simplified, isolated system|
|Sample VNF testing||Test scenarios as for the forwarder, but using a fully constructed VNF between source and destination to get a system test of performance in a real world scenario|
|Orchestration performance for the forwarder|| Test scenarios (Tempest/Rally) to measure the performance of forwarder modifications: |
* Time to create a new forwarder
* Time to add/remove ports on a forwarder
* Time to add/remove flows on a forwarder
|Forwarder focused system scale||Scale testing scenarios (Tempest/Rally) test how many VMs can be supported per forwarder|
|Platform SLA baselining||Scenario tests: Benchmarking response times, concurrent requests: Create/remove 10, 100, 1000 VNFs|