vnf_deployment_test_cases
Differences
This shows you the differences between two versions of the page.
| Both sides previous revision Previous revision Next revision | Previous revision | ||
|
vnf_deployment_test_cases [2014/10/31 02:45] Palani Chinnakannan [Use case driven Tests] |
vnf_deployment_test_cases [2014/11/21 17:22] (current) Palani Chinnakannan [VNFs Needed] |
||
|---|---|---|---|
| Line 1: | Line 1: | ||
| ======= VNF Deployment Test cases ======= | ======= VNF Deployment Test cases ======= | ||
| + | --- //[[pals@cisco.com|Palani Chinnakannan]] 2014/10/05 20:45// | ||
| + | |||
| ** An Integration & Testing project category** | ** An Integration & Testing project category** | ||
| - | ====== Introduction ====== | + | ====== Overview ====== |
| ===== Project description ===== | ===== Project description ===== | ||
| The first release of the OPNFV (Open Source Network Function Virtualization) community project targets open source component integration to form the NFVI platform for virtual network function (VNF) deployment and for NFVI platform performance measurements. A set of simple use cases are required to provide focus and drive the project to meet the following goals: | The first release of the OPNFV (Open Source Network Function Virtualization) community project targets open source component integration to form the NFVI platform for virtual network function (VNF) deployment and for NFVI platform performance measurements. A set of simple use cases are required to provide focus and drive the project to meet the following goals: | ||
| Line 17: | Line 19: | ||
| ===== Use Cases ===== | ===== Use Cases ===== | ||
| The ETSI use cases document[2] lists a wide range of use cases. After a review of this document, followed by a set of discussions with the network operators and the community, and applying the above rationale for use case selection, the following high level use cases are selected for the first phase of the OPNFV platform testing. | The ETSI use cases document[2] lists a wide range of use cases. After a review of this document, followed by a set of discussions with the network operators and the community, and applying the above rationale for use case selection, the following high level use cases are selected for the first phase of the OPNFV platform testing. | ||
| - | * Virtual Provider Edge (section 6.3) | + | * [[Virtual Provider Edge]] (section 6.3) |
| - | * EPC Network Function, MME or PGW (section 9.3) | + | * [[EPC Network Function, MME or PGW]] (section 9.3) |
| - | * IP Multimedia Subsystem | + | * [[IP Multimedia Subsystem]] (section 9.3) |
| - | * Content Delivery Network, CDN Node (Section 12.3) | + | * [[Content Delivery Network, CDN Node]] (Section 12.3) |
| - | * Home Gateway, NAT, ALG, RGW etc. (section 11.3) | + | * [[Home Gateway, NAT, ALG, RGW]] etc. (section 11.3) |
| Line 27: | Line 29: | ||
| These use cases can be achieved using open source VNFs, custom simulators those mimic the traffic patterns exhibited by the above network devices and finally using some Vendors VNF. | These use cases can be achieved using open source VNFs, custom simulators those mimic the traffic patterns exhibited by the above network devices and finally using some Vendors VNF. | ||
| - | ====== Test Methodology ====== | + | ======= Test Methodology ======= |
| + | ====== Overview ====== | ||
| This project enables the functionality and the integration testing of the OPNFV platform that is built using the open source components. In addition, the project enables performance measurement of the performance of the VNF and the OPNFV platform, which exposes the limitations and bottlenecks present in the infrastructure software and hardware as well as the VNF limitations. These efforts enable and the OPNFV platform to be designed on solid open source infrastructure as well as meet the demanding needs of network services and operations. This testing can be approached using two methodologies, the top down approach which is driven by use cases; and the bottom up approach that tests the platform functions. | This project enables the functionality and the integration testing of the OPNFV platform that is built using the open source components. In addition, the project enables performance measurement of the performance of the VNF and the OPNFV platform, which exposes the limitations and bottlenecks present in the infrastructure software and hardware as well as the VNF limitations. These efforts enable and the OPNFV platform to be designed on solid open source infrastructure as well as meet the demanding needs of network services and operations. This testing can be approached using two methodologies, the top down approach which is driven by use cases; and the bottom up approach that tests the platform functions. | ||
| Line 35: | Line 38: | ||
| ===== Platform Benchmarking driven Tests ===== | ===== Platform Benchmarking driven Tests ===== | ||
| + | Platform benchmarking (bottom up) approach identifies a basic set of VNFs for deployment and specifies a list of test cases that are used to measure the OPNFV platform characteristics as well as certain core functions of the OPNFV platform. Although, this approach can be used for platform functionality testing, the focus of these test cases will be to specify various platform components and associated metrics and detailed test plan to test these component's behavior through the measurement of the associated metrics. For example, Scaling and performance the NIC, vNIC, vSwitch, hyper-visor, CPU, Memory, storage devices etc., and their associated metrics like throughput, latency, cpu utilization, etc. | ||
| - | ===== Interface and API ===== | + | The Use case driven test cases will leverage these platform metrics measurement test cases to cover the performance related tests. |
| + | |||
| + | ====== Interface and API ====== | ||
| The project does not propose any new API specification. The project uses the standard open source component APIs from openstack, ODL, KVM etc. The project will test out these component APIs and their applicability for a stand alone NFV use cases identified above. The exact APIs set under test are [[here]] | The project does not propose any new API specification. The project uses the standard open source component APIs from openstack, ODL, KVM etc. The project will test out these component APIs and their applicability for a stand alone NFV use cases identified above. The exact APIs set under test are [[here]] | ||
| - | ===== VNF test case selection ===== | + | ====== Platform Functionality and Integration Testing ====== |
| - | + | ||
| - | List of VNF test case candidates: | + | |
| - | * Simple/Canonical: vAAA (e.g FreeRadius), ALG, NAT, vFW (e.g. IPTables), vProxy, vSIP _proxy (e.g. opensips), ... | + | |
| - | * Advanced: vIMS, vEPC, vCPE, vCDN | + | |
| - | + | ||
| - | ===== Platform Functionality and Integration Testing ===== | + | |
| | | ||
| - | ==== OPNFV Platform Integration Macro Test Plan ==== | + | ===== Macro Tests outline ===== |
| The goal of the infrastructure integration testing is to verify the integration of the infrastructure components like openstack, ODL etc. This following macro test scenarios are specified to verify this integration requirement: | The goal of the infrastructure integration testing is to verify the integration of the infrastructure components like openstack, ODL etc. This following macro test scenarios are specified to verify this integration requirement: | ||
| - Install and bring-up verification of the test system | - Install and bring-up verification of the test system | ||
| Line 54: | Line 54: | ||
| - Deployment of the Service | - Deployment of the Service | ||
| - Defining the Day 1 Configuration of the VNF | - Defining the Day 1 Configuration of the VNF | ||
| - | - Verification of the Service characteristics | + | - Verification of the Service characteristics |
| + | Details of these test cases are identified here. | ||
| + | ===== Micro Tests outline ===== | ||
| + | TBD | ||
| - | ==== OPNFV Platform Performance Measurement Macro Test Plan==== | + | ====== Platform Performance Measurement Testing====== |
| - | The goal of this Use case is to verify the delivered performance by the infrastructure for certain NFV service use cases. The macro test cases are as follows under some realistic traffic pattern | + | The performance measuring test suites focuses on the behavior of the core functionality and associated performance of the use case identified above when subjected to realistic traffic patterns associated with the use case. In this context the test suites identifies the use case level metrics, for example BGP convergence time etc., which is then mapped to platform related core component performance and associated metrics, for example, the vNIC bandwidth, PPS etc. |
| - | - Performance of a VPN Connection through a vPE NFV | + | |
| - | - Performance of a ELINE connection through a vCPE NFV | + | |
| - | - Performance of a Mobile Core Network through a vEPC NFV Service | + | |
| - | - Performance of a virtual CDN NFV ? | + | |
| ===== Traffic Profiles and Patterns===== | ===== Traffic Profiles and Patterns===== | ||
| Line 75: | Line 74: | ||
| - vNAT | - vNAT | ||
| - | The following link provides a [[list of open source]] VNF, application domain, strength weakness etc. | + | The following link provides a [[list of open source]] and vendor specific VNF, application domain, strength weakness etc. |
| + | ===== Macro Tests Outline ===== | ||
| + | The macro test suites are as follows under some realistic traffic pattern | ||
| + | - Performance of a VPN Connection through a vPE NFV | ||
| + | - Performance of a ELINE connection through a vCPE NFV | ||
| + | - Performance of a Mobile Core Network through a vEPC NFV Service | ||
| + | - Performance of the IMS core subsystem. | ||
| + | - TODO: Need more clarity here and requires revision. | ||
| - | ==== OPNFV Platform Basic Performance Measurement Micro Test Plan==== | + | ===== Micro Tests Outline===== |
| The micro test cases for this use case is built on top of the above use case. These are as follows: | The micro test cases for this use case is built on top of the above use case. These are as follows: | ||
| - VNF Orchestration and deployment time | - VNF Orchestration and deployment time | ||
| Line 86: | Line 92: | ||
| - VNF Hyper-visor performance | - VNF Hyper-visor performance | ||
| - VNF Storage performance | - VNF Storage performance | ||
| - | ==== | + | ====== Debugging, Tracing, Show CLI, Statistics ====== |
| - | Debugging and Tracing ==== | + | TBD |
| + | |||
| + | ====== Unit/Integration Test plans ====== | ||
| + | This project requires a testing platform that can provide both automated and manual testing. A certain of open source based test frameworks are available as follows: | ||
| + | [[http://robotframework.org/| Robot framework]] | ||
| + | |||
| + | [[http://www.opensourcetesting.org/functional.php| Open Source Testing]] | ||
| + | |||
| + | The community needs to determine the suitable framework. | ||
| - | Unit/Integration Test plans | ||
| Client tools developed for status shows etc. | Client tools developed for status shows etc. | ||
| Identity a list of features and functionality will be developed. | Identity a list of features and functionality will be developed. | ||
| Identify what is in or out of scope. So during the development phase, it helps reduce discussion. | Identify what is in or out of scope. So during the development phase, it helps reduce discussion. | ||
| - | Describe how the project is extensible in future | + | ====== Extensibility ====== |
| + | This project identifies the basic use cases for the first release of OPNFV. In future releases, the identified use cases will be further enhanced and developed to include other functions. In addition, additional use cases will also be included based on the community input. In order to meet this requirement, the frameworks used for testing will be generic. | ||
| + | |||
| + | |||
| + | ====== Documentation ====== | ||
| + | Basic documentation starts in this wiki link. The use cases provide additional links those specify more detailed documentation. In addition, the following Etherpad based documents provide detailed documentation. | ||
| + | |||
| + | ===== vPE Use case Doc ===== | ||
| + | The virtual Provider Edge Use case is documented [[https://etherpad.opnfv.org/p/vPE_Use_Case_Doc|here]] | ||
| + | |||
| + | ===== vEPC Use case Doc ===== | ||
| + | The virtual Evolved Packet Core use case is documented [[https://etherpad.opnfv.org/p/vEPC_Use_Case_Doc|here]] | ||
| + | ===== vIMS Use case Doc ===== | ||
| + | |||
| + | ===== vCPE Use case Doc ===== | ||
| + | |||
| + | ===== vCDN Use Case Doc ===== | ||
| - | ==== Testability ==== | + | ===== Platform performance measurement Doc ===== |
| - | Specify testing and integration like interoperability, scalability, high availablity | + | The Platform performance measurement document is documented [[https://etherpad.opnfv.org/p/Platform_Performance_Measurement_doc | here]] |
| - | What QA and test resources will be available? | + | |
| - | ==== Documentation ==== | ||
| - | API Docs | ||
| - | Functional block description | ||
| - | ==== Dependencies ==== | + | ====== Dependencies ====== |
| - No Similar project identified. | - No Similar project identified. | ||
| - The project completion requires the integration of open source components specified below: | - The project completion requires the integration of open source components specified below: | ||
| Line 112: | Line 137: | ||
| ==== Commiters and Contributors ==== | ==== Commiters and Contributors ==== | ||
| - | ==== Planned deliverable ==== | + | ====== Planned deliverable ====== |
| This project will deliver the following: | This project will deliver the following: | ||
| * Use cases document | * Use cases document | ||
| Line 118: | Line 143: | ||
| * Micro test cases driven by the Macro Test cases | * Micro test cases driven by the Macro Test cases | ||
| * Performance measurement metrics | * Performance measurement metrics | ||
| - | ==== Proposed Release Schedule ==== | + | ====== Proposed Release Schedule ====== |
| This project proposal is for the first release of the OPNFV platform. | This project proposal is for the first release of the OPNFV platform. | ||
| - | ==== References ==== | + | ====== References ====== |
| 1. [[http://portal.etsi.org/tb.aspx?tbid| ETSI NFV ISG Portal]] | 1. [[http://portal.etsi.org/tb.aspx?tbid| ETSI NFV ISG Portal]] | ||
vnf_deployment_test_cases.1414723512.txt.gz · Last modified: 2014/10/31 02:45 by Palani Chinnakannan
Page Tools
Except where otherwise noted, content on this wiki is licensed under the following license: Creative Commons Attribution 4.0 International License
