Wiki

This page is read only. You can view the source, but not change it. Ask your administrator if you think this is wrong.
==== Project Name: ====

  * Proposed name for the project: ''Yardstick''
  * Proposed name for the repository: ''yardstick''
  * Project Categories:
    * Integration & Testing

==== Project description: ====
  * This project proposes a test framework, including test cases and test stimuli, to verify the infrastructure compliance when  running VNF applications. 

==== Scope: ====

NFV Use Cases described in ETSI GS NFV 001 show a large variety of applications, each defining specific requirements and complex configuration on the underlying infrastructure and test tools. Instead of using one test-VNF application, this project aims at decomposing typical VNF work-load performance metrics into a number of characteristics/performance vectors, which each of them can be represented by distinct test-cases. This approach gives traceable results, thereby providing good fault isolation and flexibility in adding new test cases when a new challenge is identified. 

The metrics shall be divided into categories, such as:

  * Compute
  * Storage
  * Networking

The test cases might be combined and run in paralel, enabling:

  * verification of simple as well as complex scenarios 
  * verification of resource-demanding requirements such as simulating millions of flows, different traffic mixes, burstiness, fault injection

The proposed methodology includes:

  * simple test programs, which breaks down VNF application requirements into several simple test cases
  * flexible test framework, allowing introduction of new test cases when needed/new challenge discovered
  * good fault isolation, enabling early fault detection
  * test stimuli, to enable infrastructure testing such as infrastructure upgrade, infrastructure recovery
  * test framework based upon open APIs (OpenStack/Heat) for easy deployment

This project will develop additional functionality to the base system testing covered by the project "Functest", such as metrics, parallel testing, fault injection, multiple topologies, scenario testing, enabling verification of more complex test cases.

This project proposes to contribute a first implementation for evaluation, which:

  * uses standard OpenStack APIs
  * written in Python
  * webDAV file system
  * supports affinity/anti-affinity
  * orchestration using Heat templates
  * logging using the facility provided with Python
  * documentation
  * test case examples
  * can be run as stand-alone
 
Candidates for functionality to be developed:

  * Result presentation
  * File System
  * Simplified API
  * Additional Networking topologies
  * Additional test cases and examples
  * Conformance testing

Presentation

{{:project_proposals:yardstick_proposal.pptx|}}
    

==== Testability: ====

  * To be integrated in OPNFV Continuous Integration testing. 

==== Dependencies: ====

  * The test framework is hardware, VNF application, operator and vendor independent.
  * Current limitation: OpenStack.
  * Yardstick will consider outputs of the ETSI ISG NFV about testing an NFV environment. Only public documents can be used, so Yardstick might need to adapt the timeplan for the deliverables where output from ongoing work, e.g. from TST working group needs to be considered, accordingly.
  * Since verification as done in Yardstick has to prepare the ground for any certification activity, the Yardstick project is dependent on the Certification & Compliance Committee and will put those issues on high priority that are needed for any certification activities that will be needed for certification or compliance if there will be a decision to engage in this.

==== Committers and Contributors: ====

  * Name and affiliation of the project leader:
    * Ana Cunha (Ericsson): ana.cunha@ericsson.com
  * Names and affiliations of the committers
    * Ana Cunha (Ericsson): ana.cunha@ericsson.com
    * Leif Johansson (Ericsson): leif.y.johansson@ericsson.com
    * Hans Feldt (Ericsson): hans.feldt@ericsson.com
    * Jörgen Karlsson (Ericsson): jorgen.w.karlsson@ericsson.com
    * Houjingwen (Huawei): houjingwen@huawei.com 
    * Trevor Cooper (Intel): trevor.cooper@intel.com
    * Wenjing Chu (DELL): Wenjing_Chu@DELL.com
  * Names and affiliations of other contributors
    * Morgan Richomme (Orange): morgan.richomme@orange.com
    * Dr. Rebecca Steinert (SICS): rebste@sics.se
    * Fatih Degirmenci (Ericsson): fatih.degirmenci@ericsson.com
    * Pierre Lynch (Ixia): PLynch@ixiacom.com
    * Julien Zhang (ZTE) :zhang.jun3g@zte.com.cn

 
==== Planned deliverables ====
 
  * Test framework
  * Test cases examples
  * Reference documentation

==== Proposed Release Schedule: ====

  * OPNFV Release R2
Wiki

User Tools

Site Tools

Page Tools
