• Proposed name for the project: vPE Performance testing
• Proposed name for the repository: opnfv_vPE_perf_test_repo
• Project Categories: Integration & Testing

Project “vPE Performance testing” will provide comprehensive testing methodology, test suites and test cases to test and the performance of the OPNFV platform under the vPE L3VPN over MPLS use case.

This project is focused on measuring the performance of the different OPNFV platform components like the compute server, hyper-visor, Open vSwitch, Linux Bridge etc., that are involved in the control and data path of the vPE L3 VPN use case. The focus of the performance measurement will be on the following aspects when the vPE is subjected to realistic traffic patterns like that of a real world PE router:

• The performance of the components involved in the vPE data path under traffic created by the CE to CE data, CE-to-Internet flows.
• The performance of the components involved in the vPE control path under traffic created by bfd, route programming, QOS programming)

This project will develop test suites that cover detailed vPE L3VPN performance measurement test cases, test methodologies and platform configurations which will be documented and maintained in a repository for use by other OPNFV testing projects and the community in general.

Approach

The project targets testing of the OPNFV platform in a hosted test-bed environment (i.e. using the OPNFV test labs world wide). The project will leverage the "OPNFV: Base system Functionality testing" project and will leverage the infrastructure, test tools, automation framework delivered bu this project.

The picture below outlines the main test topology used in the vPE performance testing.

vPE Performance test project will deliver a performance testing system along with a set of test traffic patterns, performance measuring metrics, test suites and test cases to test the performance of OPNFV platform when used in the vPE use case under the specific L3VPN over MPLS. The performance measurement is targeted at the following entities in the OPNFV Platform. NOTE: These tests are performed on the VNF, however, the metrics measured under these traffic patterns are on the OPNFV platform that are specified further down below.

• Control Plane setup as a black box with focus on OPNFV platform characteristics.
  • vPE VNF bring-p and set-up time.
  • vPE VNF Destroy and tear down time.
  • VNF Global Routing Convergence time
  • VNF FIB download and ready for forwarding time.
  • L3 VPN VRF bring-up and routing convergence time.
  • Filters (QOS and ACL) programming time.
  • BFD Setup time
• Data Plane performance
  • Data Path non stop forwarding for 24 hours
  • Mix of Traffic Patterns (small, medium and large packets only, iMix traffic etc.)
  • Traffic with mix of IPV4 packet prefixes (/8, /16, /24, /30) at the prescribed ratios.
  • Traffic subjected to packet filters on a per VRF with a maximum of 40 Filters per VRF
  • Traffic subjected QOS treatment with four classes of traffic unclassified, RT, AF, BE.

The following metrics are measured under the above testcases and traffic patterns. Please refer to the follwoing figure to get more clarity:

• Black Box Metrics (VNF as a Black Box)
  • Data NIC bandwidth (bits per second), PPS (Packets Per Second), BPS (Bytes per second), Latency(measured between the ingress and egress of the Physical NICs), Jitter
  • Open vSwitch External Bridge metrics (same as the above).
  • Open vSwitch Integration Bridge metrics (same as the above).

The project requires the following components:

• OPNFV Lab setup with complete access to a set of Bare metal servers for Controller and Compute nodes (minimum configuration: 1x installer server, 2x controller node servers, 3x compute node servers); associated switches and routers.
• OPNFV platform software bundle from the repository that includes several upstream software components.
• OPNFV "Get Started!" software bundles that includes the installer
• Open Source VNF Open-WRT
• Open Source VNF Snort
• Open Source Apache Web Server in Ubuntu 14.04
• Open Source HAProxy
• Tempest framework and Robot Framework

The following link provides the detailed test suites and drills down into the test suites/test cases to cover the functional testing.

Function Test Suites

The above test suites/cases are automated via Robot/Jenkins):

• The project is a contributor to project “Octopus”.
• The project leverages "Boot Strap/Get Started"
• The project relies on the following upstream projects:
  • OpenStack Juno release: Components: Nova, Glance, Keystone, Horizon, Neutron, Ceilometer, Heat
  • OpenDaylight Helium release: Components: MDSAL, OVSDB, RESTCONF, ML2 plugin/ODL Neutron drivers
  • Installer: TBD
  • Puppet (for instance configuration)
  • QEMU/KVM
  • OpenWRT (as example VNF - for routing, firewall, NAT)
  • Snort (as example VNF - for IDS)
  • Linux/Centos 7 distribution

• Comitters
  • Palani Chinnakannan (pals@cisco.com)
  • Trevor Cooper (trevor.cooper@intel.com)
  • Prabu.K (Prabu.Kuppuswamy@spirent.com)
  • Ian Wells (iawelss@cisco.com)
  • Qiao Fu (fuqiao@chinamobile.com)
  • Chitti Nimmagadda (chitti.nimmagadda@ericsson.com)
• Contributors
  • Palani Chinnakannan (pals@cisco.com)
  • Trevor Cooper (trevor.cooper@intel.com)
  • Frank Brockners (fbrockne@cisco.com)
  • Prabu.K (Prabu.Kuppuswamy@spirent.com)
  • Ian Wells (iawelss@cisco.com)
  • Qiao Fu (fuqiao@chinamobile.com)
  • Sajeev Manikkoth (sajeevmanikkoth@gmail.com)
  • Chitti Nimmagadda (chitti.nimmagadda@ericsson.com)

Project deliverable: The project delivers the following components:

• Documentation of Test Suites and test cases that cover the OPNFV platform functionality testing including pass/fail criteria.
• Test software and scripts for testing OPNFV Platform functionality that is essentially broken down into two main suites as follows:
  • OPNFV Platform Smoke test suite: A sanity testing suite for basic verification of the platform.
  • OPNFV Platform Regression test suite: A comprehensive collection of detailed test suites.
• Automated Test Framework based on Tempest/Robot/Jenkins, with necessary scripts and tools to automatically test and verify OPNFV functionality

OPNFV release #1.