======= Bootstrap/Get started - Candidate work items ======= ===== Task management ===== Complementary to this wiki, we'll use [[https://etherpad.opnfv.org/p/bgs]] to track activity. ===== System definition ===== The test environment integrates the following components and is expected to deploy to bare metal as well as to a virtual environment. All setup and install procedures will be automated (no manual setup/testing). The deploy/test cycle is intended to be run periodically by a job scheduler (Jenkins) and produce a pass/fail result. ** Outline of the environment ** {{:get_started:bgs-environment.png|}} ** Components ** See [[get_started:get_started_system_state|Target System State]] for a complete list of components on versions. * //Linux/Centos (multiple control nodes, multiple compute nodes)// * //OpenStack Juno Release// * Nova, Glance, Ceilometer Neutron, Keystone, Horizon, Heat * MySQL, RabbitMQ, Pacemaker cluster stack, Corosync * Tempest, Rally (for testing) * //OpenDaylight Helium Release// * MDSAL, Clustering, Restconf, OVSDB, OpenFlow, SFC, GBP, ML2-plugin, Netconf * //Hypervisor:// * KVM, QEMU * //Forwarder// * OVS * //Installer // * TBD * Puppet (version …) * //Example VNFs// * vRouter based on OpenWRT (version 14.07 – barrier braker) * vIDS based on SNORT (version 2.9.7.0) * //Automation// * Robot * Jenkins ---- ===== Use cases for system level test (initial focus) ===== The following set of use-cases will guide the testing performed as part of Bootstrap/Get-started. The project does not intent to provide a comprehensive coverage of the particular use-cases and all associated testing scenarios. We just need a few VNFs that we can run on the environment Bootstrap/Get-started provides. Having a few example VNFs (rather than only one) is to ensure that test scenarios are kept generic wherever possible. Use-case specific test projects are expected to provide proper and full coverage for specific use-cases. ^ Use-Case ^ Description ^ Interested contributors ^ | Simple VNF: vRouter | Bring up one or multiple instances of a simple virtual router. OpenWRT will be used as virtual router. | Joseph Gasparakis | | Simple VNF: vIDS | Bring up one or multiple instances of a service appliance. Snort/IDS will be used as service appliance. | Joseph Gasparakis | | SFC | Service chaining using open source components: Firewall and IDS (OpenWRT as simple firewall, Snort as IDS) | | ---- ===== Candidate work items ===== This section lists tasks and test cases which Bootstrap/Get-started aspires to perform. It is expected that the scenario tests fold into more specific test projects (e.g. for system level functionality testing, specific VNF use-case tests, performance tests etc.) - which are in the process of being defined. ==== System setup ==== ^ Test/Task ^ Description ^ Interested contributors ^ | Environment setup | Setup base hardware environment (servers, network connectivity) | Linux Foundation | | Server boot up | PXE boot the servers and prepare them with base OS to load OPNFV components. | | | Install OPNFV components | Automatic install OpenStack control and compute nodes (with an HA setup). \\ Automatic install OpenDaylight (with an HA setup). \\ Automatic install of test harness components such as Tempest, Rally.| | ==== System base functionality testing ==== ^ Test/Task ^ Description ^ Interested contributors ^ | Components function test: OpenStack | Ensure OpenStack components are installed correctly and functioning. Leverage [[http://docs.openstack.org/developer/tempest/index.html|OpenStack Tempest]] and [[https://wiki.openstack.org/wiki/Rally|OpenStack Rally]]. Includes smoke, baremetal, compute, data_processing, identity, image, network, object_storage, orchestration, telemetry, and volume testing ([[https://github.com/openstack/tempest/tree/master/tempest/scenario|current list]] of tempest scenario tests available). See [[http://logs.openstack.org/78/134978/1/check/check-tempest-dsvm-neutron-full/7ebce20/logs/testr_results.html.gz|example]] for the result of a tempest run. | | | Components function test: OpenDaylight/Networking | Ensure OpenDaylight components are installed correctly and functioning. Check whether relevant Karaf modules are loaded and functioning correctly. | | | Scenario testing: Basic system testing | Basic system testing leverages Tempest Scenario testing with Openstack deployment configured to use Neutron/ML2 managed networking supplied by OpenDaylight.\\ * Basic network ops: Test network connectivity to a VM via a tenant network, a public network, or both. \\ * Advanced server networking: Test VM connectivity after some advanced instance operations executed: start/stop instance, reboot instance, migrate instance, etc. Tests prioritised by relevance to NFV workloads. \\ * Install and removal of ODL Karaf features on running instance. \\ * Security groups: Verify cross tenant connectivity - basic as well as after resizing an instance etc. | | | Scenario testing: High-availability testing | HA testing leveraging Tempest Scenario testing - component and physical infrastructure failures.\\ Test component HA/failure-recovery for those components that support it: \\ * Nova scheduler \\ * Nova conductor \\ * Cinder scheduler \\ * Neutron server \\ * Heat engine \\ Test hardware and system failures by emulating failure scenarios \\ * dead server \\ * dead switch \\ * dead port \\ * dead disk \\ * full disk | | ==== System VNF-management operations testing ==== ^ Test/Task ^ Description ^ Interested contributors ^ | Simple VNF creation | Create VM images for vRouter (from OpenWRT) and vIDS (from Snort) for testing purposes | Frank | | Simple VNF hosting: base | Scenario test using Tempest to create and configure a simple VNF: Create 3 VMs: 2 hosts and one instance of a vRouter (OpenWRT) with basic configuration. Test whether vRouter is reachable (within its domain) from the host and can pass packets between the hosts. | | | Simple VNF hosting: Scale/Idempotence testing | Scenario test using Tempest to create/remove n-instances of a vRouter. (Test requires a simple VNFM which in this case will be provided by a script) | | | Simple VNF hosting: Sequencing | Scenario test using Tempest to investigate system behavior in case components are not triggered/loaded in the correct sequence (i.e. VNFM would behave wrongly) | | | Simple VNF hosting: Resource control | Scenario test using Tempest to deploy n instances of vRouter and check for allocated vCPUs, check vCPU pinning. | | | Simple VNF hosting: Fault management | Scenario test using Tempest to test detection of VNF failure (vRouter) | | | Simple VNF hosting: Service upgrade | Scenario test using Tempest to replace an existing VNF (vRouter) with a new revision | | | SFC: base | Scenario test using Tempest to create a service chain from a set of open-source VNFs: Firewall and IDS: OpenWRT and Snort. Basic functionality test of the service chain. | | | Eventing/Statistics | Scenario tests to test capturing events and statistics from different system components (VNFs, ODL, OpenStack) | | ==== Basic system performance benchmark and scale numbers ==== ^ Test/Task ^ Description ^ Interested contributors ^ | Virtual forwarder testing | Test scenario descriptions for virtual forwarders (e.g. OVS) deployed by the OPNFV platform. Forwarding performance testing between two or more hosts: \\ * Different traffic profiles (large packets, small packets, IMIX - unicast, multicast) and network configurations (QoS, different tunnel types, etc.) \\ * bare metal deployment (virtual forwarder directly deployed on host); Single/multiple virtual forwarders on the same instance \\ * virtual environments (VM based deployment); Single/multiple virtual forwarders on the same instance | | | Sample VM testing | Test scenarios as for the forwarder, but using a VNF VM between source and destination to get realistic idea of performance in a simplified, isolated system | | | Sample VNF testing | Test scenarios as for the forwarder, but using a fully constructed VNF between source and destination to get a system test of performance in a real world scenario | | | Orchestration performance for the forwarder | Test scenarios (Tempest/Rally) to measure the performance of forwarder modifications: \\ * Time to create a new forwarder \\ * Time to add/remove ports on a forwarder \\ * Time to add/remove flows on a forwarder | | | Forwarder focused system scale | Scale testing scenarios (Tempest/Rally) test how many VMs can be supported per forwarder | | | Platform SLA baselining | Scenario tests: Benchmarking response times, concurrent requests: Create/remove 10, 100, 1000 VNFs | |