Wiki

This page is read only. You can view the source, but not change it. Ask your administrator if you think this is wrong.
======= VNF Deployment Test cases =======
 --- //[[pals@cisco.com|Palani Chinnakannan]] 2014/10/05 20:45//

**    An Integration & Testing project category**
====== Overview ======
===== Project description =====
The first release  of the OPNFV (Open Source Network Function Virtualization) community project targets open source component integration to form the NFVI platform for virtual network function (VNF) deployment and for NFVI platform performance measurements. A set of simple use cases are required to provide focus and drive  the project to meet the following goals:
    * OPNFV Platform functional testing
    * OPNFV Platform performance measurements
==== OPNFV Platform functional testing ====
The  use case(s) selected or a subset of them provide a comprehensive set of test cases and  tests required to verify the OPNFV Platform functionality. The OPNFV platform functionality includes  all the functions provided by the Virtualization Infrastructure Manager (VIM) and the Network Function Virtualization Infrastructure (NFVI) components as specified by the ETSI NFV ISG Specifications [1]. The functional test cases and tests identified shall verify the __**integration**__ of the components like Openstack, ODL, KVM etc., to form a cohesive OPNFV Platform, to test such a platform on its ability to meet the above referenced functions and finally identify gaps in the functionality of the platform in order initiate further projects to address those gaps.

==== OPNFV Platform Performance Measurement====
The use case(s) selected must also provide adequate testable scenarios, test cases and tests to enable the measurement of the OPNFV platform performance in terms of commonly used network metrics like bandwidth, latency, number of subscribers, number of connections etc. The performance measurement enables the operators to determine, whether the OPNFV platform meets the requirement for deploying VNF in lieu of PNFs and the associated costs. These measurements also enables the community to identify performance limiting hardware and software entities in the system and provide solutions to improve the performance of the system. Finally, it helps the different vendors to optimize their VNF offerings and also provide better performing components within the platform. 

===== Rationale for Use case Selection =====
The use case(s) identified, in addition to meeting the above specified goals, should be representative of the traffic profiles experienced by a wide spectrum of network services and applications commonly deployed in the internet (IP Network). A network service or function is characterized by how it behaves in the IP network under certain traffic conditions ( steady/bursting of small packets, large packets, mix of packets etc.), load (number of subscribers, service traffic type), demand etc., which are collectively called a traffic profile.

===== Use Cases =====
The ETSI use cases document[2] lists a wide range of use cases. After a review of this document, followed by a set of discussions with the network operators and the community, and applying the above rationale for use case selection, the following high level use cases are selected for the first phase of the OPNFV platform testing. 
      * [[Virtual Provider Edge]] (section 6.3)
      * [[EPC Network Function, MME or PGW]] (section 9.3)
      * [[IP Multimedia Subsystem]] (section 9.3)
      * [[Content Delivery Network, CDN Node]] (Section 12.3)
      * [[Home Gateway, NAT, ALG, RGW]] etc. (section 11.3)


For more details about these use cases, please refer to the above reference:
These use cases can be achieved using open source VNFs, custom simulators those mimic the traffic patterns exhibited by the above network devices and finally using some Vendors VNF.  

======= Test Methodology =======
====== Overview ======

This project enables the functionality and the integration testing of the OPNFV platform that is built using the open source components.  In addition, the project enables performance measurement of the performance of the VNF and the OPNFV platform, which exposes the limitations and bottlenecks present in the infrastructure software and hardware as well as the VNF limitations. These efforts enable and the OPNFV platform to be designed on solid open source infrastructure as well as meet the demanding needs of network services and operations. This testing can be approached using two methodologies, the top down approach which is driven by use cases; and the bottom up approach that tests the platform functions.

===== Use case driven Tests =====
Use case driven testing starts from the high level use cases identified above and breaks down the use case into a basic core function in its minimalistic form and provides the necessary topology, traffic patterns etc. For example, a minimalistic core function of the vPE VNF would be to provide the basic PE router functionality to act as an L3VPN aggregation point for a set of CE routers. The use case driven test cases would define  a suite of basic tests required to deploy the vPE VNF, configure it to support a set of CE devices, support basic L3VPN networks for each CE device and provide connectivity to the core P router. Then it would further define performance related test suites that subject the vPE VNF to different traffic profiles as seen in a real live network. These two suites of test cases are defined in detail to identify the core functions of the OPNFV platform for functionality testing as well as the performance metrics of the platform that needs to be measured.  This approach is detailed in the subsections associated with each use case. Please click on the links associated with each use case. 

===== Platform Benchmarking driven Tests =====
Platform benchmarking (bottom up) approach identifies a basic set of VNFs for deployment and specifies a list of test cases that are used to measure the OPNFV platform characteristics as well as certain core functions of the OPNFV platform. Although, this approach can be used for platform functionality testing, the focus of these test cases will be to specify various platform components and associated metrics and detailed test plan to test these component's behavior through the measurement of the associated metrics. For example, Scaling and performance the NIC, vNIC, vSwitch, hyper-visor, CPU, Memory, storage devices etc., and their associated metrics like throughput, latency, cpu utilization, etc.

The Use case driven test cases will leverage these platform metrics measurement test cases to cover the performance related tests. 

====== Interface and API ======
The project does not propose any new API specification. The project uses the standard open source component APIs from openstack, ODL, KVM etc.  The project will test out these component APIs and their applicability for a stand alone NFV use cases identified above. The exact APIs set under test are [[here]]

====== Platform Functionality and Integration Testing ======
    
===== Macro Tests outline =====
The goal of the infrastructure integration testing is to verify the integration of the infrastructure components like openstack, ODL etc. This following macro test scenarios are specified to verify this integration requirement:
  - Install and bring-up verification of the test system
  - On-boarding of the Service VNF Descriptor
  - Defining  the Day 0 Configuration for the VNF
  - Deployment of the Service
  - Defining the Day 1 Configuration of the VNF 
  - Verification of the Service characteristics
Details of these test cases are identified here. 
===== Micro Tests outline =====
TBD

====== Platform  Performance Measurement Testing======
The performance measuring test suites focuses on the behavior of the core functionality and associated performance of the use case identified above when subjected to realistic traffic patterns associated with the use case.  In this context the test suites identifies the use case level metrics, for example BGP convergence time etc., which is then mapped to platform related core component performance and associated metrics, for example, the vNIC bandwidth, PPS etc. 

===== Traffic Profiles and Patterns=====
  - A PE router traffic pattern 
  - A CPE Device initiated traffic Pattern over a ELINE
  - Mobile Core traffic pattern
  - CDN network traffic.

===== VNFs Needed =====
  - vPE 
  - vPGW
  - vFW
  - vNAT

The following link provides a [[list of open source]] and vendor specific VNF, application domain, strength weakness etc.

===== Macro Tests Outline =====
The macro test suites are as follows under some realistic traffic pattern
  - Performance of a VPN Connection through a vPE NFV
  - Performance of a ELINE connection through a vCPE NFV
  - Performance of a Mobile Core Network through a vEPC NFV Service
  - Performance of the IMS core subsystem.
  - TODO: Need more clarity here and requires revision.

===== Micro Tests Outline=====
The micro  test cases for this use case is built on top of the above use case. These are as follows:
  - VNF Orchestration and deployment time
  - VNF boot up time
  - VNF Network performance
  - VNF Compute performance
  - VNF Hyper-visor performance
  - VNF Storage performance
====== Debugging, Tracing, Show CLI, Statistics ======
TBD

====== Unit/Integration Test plans ======
This project requires a testing platform that can provide both automated and manual testing. A certain of open source based test frameworks are available as follows:
[[http://robotframework.org/| Robot framework]]

[[http://www.opensourcetesting.org/functional.php| Open Source Testing]]

The community needs to determine the suitable framework.

Client tools developed for status shows etc.
Identity a list of features and functionality will be developed.
Identify what is in or out of scope. So during the development phase, it helps reduce discussion.
====== Extensibility ======
This project identifies the basic use cases for the first release of OPNFV. In future releases, the identified use cases will be further enhanced and developed to include other functions. In addition, additional use cases will also be included based on the community input. In order to meet this requirement, the frameworks used for testing will be generic.


====== Documentation ======
Basic documentation starts in this wiki link. The use cases provide additional links those specify more detailed documentation. In addition, the following Etherpad based documents provide detailed documentation.

===== vPE Use case Doc =====
The virtual Provider Edge Use case is documented [[https://etherpad.opnfv.org/p/vPE_Use_Case_Doc|here]]

===== vEPC Use case Doc =====
The virtual Evolved Packet Core use case is documented [[https://etherpad.opnfv.org/p/vEPC_Use_Case_Doc|here]]
===== vIMS Use case Doc =====

===== vCPE Use case Doc =====

===== vCDN Use Case Doc =====

===== Platform performance measurement Doc =====
The Platform performance measurement document is documented [[https://etherpad.opnfv.org/p/Platform_Performance_Measurement_doc | here]]




====== Dependencies ======
      - No Similar project identified.
      - The project completion requires the integration of open source components specified below:
      - Depends on the Continuous Integration Plan
      - Depends on the Hardware and Software Integration Plan
==== Commiters and Contributors ====

====== Planned deliverable ======
This project will deliver the following:
  * Use cases document
  * Macro Test cases driven by the use cases
  * Micro test cases driven by the Macro Test cases
  * Performance measurement metrics
====== Proposed Release Schedule ======
This project proposal is for the first release of the OPNFV platform.
====== References ======
1. [[http://portal.etsi.org/tb.aspx?tbid| ETSI NFV ISG Portal]]

2. ETSI GS NFV 001 V1.1.1: Network Function Virtualization; Use cases
Wiki

User Tools

Site Tools

Page Tools
