vpe_performance_testing
Differences
This shows you the differences between two versions of the page.
| Both sides previous revision Previous revision Next revision | Previous revision | ||
|
vpe_performance_testing [2014/12/06 21:12] Palani Chinnakannan |
vpe_performance_testing [2014/12/12 17:32] Palani Chinnakannan |
||
|---|---|---|---|
| Line 13: | Line 13: | ||
| * The performance of the components involved in the vPE control path under traffic created by bfd, route programming, QOS programming) | * The performance of the components involved in the vPE control path under traffic created by bfd, route programming, QOS programming) | ||
| - | Edit below not done. | + | This project will develop test suites that cover detailed vPE L3VPN performance measurement test cases, test methodologies and platform configurations which will be documented and maintained in a repository for use by other OPNFV testing projects and the community in general. |
| - | + | ||
| - | This project will develop test suites that cover detailed functional test cases, test methodologies and platform configurations which will be documented and maintained in a repository for use by other OPNFV testing projects and the community in general. Developing test suites will also help lay the foundation for a test automation framework that in future can be used by the continuation integration (CI) project (Octopus). We envisage that certain VNF deployment use-cases could be automatically tested as an optional step of the CI process. | + | |
| Approach | Approach | ||
| - | The project targets testing of the OPNFV platform in a hosted test-bed environment (i.e. using the OPNFV test labs world wide). It will leverage output of the "Get started!" project. | + | The project targets testing of the OPNFV platform in a hosted test-bed environment (i.e. using the OPNFV test labs world wide). The project will leverage the "OPNFV: Base system Functionality testing" project and will leverage the infrastructure, test tools, automation framework delivered bu this project. |
| - | The picture below outlines the main setup of “OPNFV – Base system functionality testing" project. | + | The picture below outlines the main test topology used in the vPE performance testing. |
| - | + | {{ :basic_vpe_topo.png?200 |}} | |
| - | {{opnfv-tester2.png|}} | + | |
| ==== Scope ==== | ==== Scope ==== | ||
| + | vPE Performance test project will deliver a performance testing system along with a set of test traffic patterns, performance measuring metrics, test suites and test cases to test the performance of OPNFV platform when used in the vPE use case under the specific L3VPN over MPLS. The performance measurement is targeted at the following entities in the OPNFV Platform. | ||
| + | **NOTE: These tests are performed on the VNF, however, the metrics measured under these traffic patterns are on the OPNFV platform that are specified further down below.** | ||
| + | * Control Plane setup as a black box with focus on OPNFV platform characteristics. | ||
| + | * vPE VNF bring-p and set-up time. | ||
| + | * vPE VNF Destroy and tear down time. | ||
| + | * VNF Global Routing Convergence time | ||
| + | * VNF FIB download and ready for forwarding time. | ||
| + | * L3 VPN VRF bring-up and routing convergence time. | ||
| + | * Filters (QOS and ACL) programming time. | ||
| + | * BFD Setup time | ||
| + | * Data Plane performance | ||
| + | * Data Path non stop forwarding for 24 hours | ||
| + | * Mix of Traffic Patterns (small, medium and large packets only, iMix traffic etc.) | ||
| + | * Traffic with mix of IPV4 packet prefixes (/8, /16, /24, /30) at the prescribed ratios. | ||
| + | * Traffic subjected to packet filters on a per VRF with a maximum of 40 Filters per VRF | ||
| + | * Traffic subjected QOS treatment with four classes of traffic unclassified, RT, AF, BE. | ||
| - | “OPNFV – Base system functionality testing” will deliver a functional testing framework along with a set of test suites and test cases to test and verify the functionality OPNFV platform. The testing framework (tools, test-cases, etc.) are also intended to be used by the CI framework for the purpose of qualifying the OPNFV platform on bare metal servers. In this context, OPNFV Tester will use open source VNF components. Functional testing includes | + | The following metrics are measured under the above testcases and traffic patterns. Please refer to the follwoing figure to get more clarity: |
| - | * Testing the installer, setup and the configuration of the OPNFV platform hardware and software. | + | * Black Box Metrics (VNF as a Black Box) |
| - | * Testing the basic VIM functionality that includes tenant, user CRUD operations, VNF Image CRUD operations etc. | + | * Data NIC bandwidth (bits per second), PPS (Packets Per Second), BPS (Bytes per second), Latency(measured between the ingress and egress of the Physical NICs), Jitter |
| - | * Testing the VIM functionality to support VNF life cycle management (create, modify, grow, shrink, destroy) | + | * Open vSwitch External Bridge metrics (same as the above). |
| - | * Testing the VIM functionality to support basic VNF network connectivity | + | * Open vSwitch Integration Bridge metrics (same as the above). |
| - | * Testing the inter working between the VIM and the SDN controller. | + | * VNF Packet Ingress Tap Interface metrics (same as the above) |
| - | * Testing the NFVI functionality as a black box to ensure that it meets the VIM requirements. | + | * VNF Packet Egress Tap Interface metrics (same as the above) |
| + | * VNF Threads CPU Utilization, Average, Mix, Max during the test interval (per test case) | ||
| + | * VNF Memory Utilization, Average, Min, Max during the test interval. | ||
| + | * Cache Performance at the HOST OS(hits, misses) | ||
| + | * Context Switches Host OS (global) | ||
| + | * Hypervisor VM Entry/Exits | ||
| + | * VNF White Box metrics | ||
| + | * Applications inside the VNF (Routing Protocols, BFD, FIBMgr etc.,) CPU utilization (Min, Max, Average) | ||
| + | * Fib forwarding performance for different traffic mixes, (Min, Average, Max) | ||
| + | * Per Application Memory Utllization | ||
| + | * Cache ? | ||
| + | TBD: Add a figure detailing the vswitch bridges etc. | ||
| The project requires the following components: | The project requires the following components: | ||
| Line 41: | Line 64: | ||
| * OPNFV platform software bundle from the repository that includes several upstream software components. | * OPNFV platform software bundle from the repository that includes several upstream software components. | ||
| * OPNFV "Get Started!" software bundles that includes the installer | * OPNFV "Get Started!" software bundles that includes the installer | ||
| - | * Open Source VNF Open-WRT | + | * OPNFV: Basic Functionality Testing project software bundles, Test suites and test cases. |
| - | * Open Source VNF Snort | + | * Tempest framework and Robot Framework |
| - | * Open Source Apache Web Server in Ubuntu 14.04 | + | * Third Party vPE VNFs |
| - | * Open Source HAProxy | + | |
| - | * Tempest framework and Robot Framework | + | |
| ==== Functional Testing covered ==== | ==== Functional Testing covered ==== | ||
| Line 58: | Line 78: | ||
| ==== Dependencies ==== | ==== Dependencies ==== | ||
| - | * The project is a contributor to project “Octopus”. | + | * The project leverages "Boot Strap/Get Started" and the "OPNFV Basic Functionality Testing Project" |
| - | * The project leverages "Boot Strap/Get Started" | + | |
| * The project relies on the following upstream projects: | * The project relies on the following upstream projects: | ||
| * OpenStack Juno release: Components: Nova, Glance, Keystone, Horizon, Neutron, Ceilometer, Heat | * OpenStack Juno release: Components: Nova, Glance, Keystone, Horizon, Neutron, Ceilometer, Heat | ||
| Line 77: | Line 96: | ||
| * Prabu.K (Prabu.Kuppuswamy@spirent.com) | * Prabu.K (Prabu.Kuppuswamy@spirent.com) | ||
| * Ian Wells (iawelss@cisco.com) | * Ian Wells (iawelss@cisco.com) | ||
| - | * Qiao Fu (fuqiao@chinamobile.com) | ||
| * Chitti Nimmagadda (chitti.nimmagadda@ericsson.com) | * Chitti Nimmagadda (chitti.nimmagadda@ericsson.com) | ||
| * Contributors | * Contributors | ||
| Line 85: | Line 103: | ||
| * Prabu.K (Prabu.Kuppuswamy@spirent.com) | * Prabu.K (Prabu.Kuppuswamy@spirent.com) | ||
| * Ian Wells (iawelss@cisco.com) | * Ian Wells (iawelss@cisco.com) | ||
| - | * Qiao Fu (fuqiao@chinamobile.com) | ||
| * Sajeev Manikkoth (sajeevmanikkoth@gmail.com) | * Sajeev Manikkoth (sajeevmanikkoth@gmail.com) | ||
| * Chitti Nimmagadda (chitti.nimmagadda@ericsson.com) | * Chitti Nimmagadda (chitti.nimmagadda@ericsson.com) | ||
| Line 93: | Line 110: | ||
| The project delivers the following components: | The project delivers the following components: | ||
| - | * Documentation of Test Suites and test cases that cover the OPNFV platform functionality testing including pass/fail criteria. | + | * Documentation of Test Suites and test cases that cover the vPE peformance testing including all the metrics measured and various graphs. |
| - | * Test software and scripts for testing OPNFV Platform functionality that is essentially broken down into two main suites as follows: | + | * Test software and scripts for testing vPE Performance |
| - | * OPNFV Platform Smoke test suite: A sanity testing suite for basic verification of the platform. | + | * Automated Test Framework based on Tempest/Robot/Jenkins |
| - | * OPNFV Platform Regression test suite: A comprehensive collection of detailed test suites. | + | * Identify Key components in the OPNFV platform that includes further development and improvement to make OPNFV a carrier class platform for hosting VNFs |
| - | * Automated Test Framework based on Tempest/Robot/Jenkins, with necessary scripts and tools to automatically test and verify OPNFV functionality | + | |
| === Proposed Release Schedule ==== | === Proposed Release Schedule ==== | ||
vpe_performance_testing.txt · Last modified: 2014/12/12 17:32 by Palani Chinnakannan
Page Tools
Except where otherwise noted, content on this wiki is licensed under the following license: Creative Commons Attribution 4.0 International License
