Wiki

• Proposed name for the project: Yardstick
• Proposed name for the repository: yardstick
• Project Categories:
  • Integration & Testing

• This project proposes a test framework, including test cases and test stimuli, to verify the infrastructure compliance when running VNF applications.

NFV Use Cases described in ETSI GS NFV 001 show a large variety of applications, each defining specific requirements and complex configuration on the underlying infrastructure and test tools. Instead of using one test-VNF application, this project aims at decomposing typical VNF work-load performance metrics into a number of characteristics/performance vectors, which each of them can be represented by distinct test-cases. This approach gives traceable results, thereby providing good fault isolation and flexibility in adding new test cases when a new challenge is identified.

The metrics shall be divided into categories, such as:

• Compute
• Storage
• Networking

The test cases might be combined and run in paralel, enabling:

• verification of simple as well as complex scenarios
• verification of resource-demanding requirements such as simulating millions of flows, different traffic mixes, burstiness, fault injection

The proposed methodology includes:

• simple test programs, which breaks down VNF application requirements into several simple test cases
• flexible test framework, allowing introduction of new test cases when needed/new challenge discovered
• good fault isolation, enabling early fault detection
• test stimuli, to enable infrastructure testing such as infrastructure upgrade, infrastructure recovery
• test framework based upon open APIs (OpenStack/Heat) for easy deployment

This project proposes to contribute a first implementation for evaluation, which:

• uses standard OpenStack APIs
• written in Python
• webDAV file system
• supports affinity/anti-affinity
• orchestration using Heat templates
• logging using the facility provided with Python
• documentation
• test case examples
• can be run as stand-alone

Candidates for functionality to be developed:

• Result presentation
• File System
• Simplified API
• Additional Networking topologies
• Additional test cases and examples
• Conformance testing

Presentation

yardstick_proposal.pptx

• To be integrated in OPNFV Continuous Integration testing.

• The test framework is hardware, VNF application, operator and vendor independent
• Current limitation: OpenStack

• Name and affiliation of the project leader:
  • Ana Cunha (Ericsson): ana.cunha@ericsson.com
• Names and affiliations of the committers
  • Ana Cunha (Ericsson): ana.cunha@ericsson.com
  • Leif Johansson (Ericsson): leif.y.johansson@ericsson.com
  • Hans Feldt (Ericsson): hans.feldt@ericsson.com
  • Jörgen Karlsson (Ericsson): jorgen.w.karlsson@ericsson.com
• Names and affiliations of other contributors
  • Morgan Richomme (Orange): morgan.richomme@orange.com
  • Dr. Rebecca Steinert (SICS): rebste@sics.se
  • Fatih Degirmenci (Ericsson): fatih.degirmenci@ericsson.com

• Test framework
• Test cases examples
• Reference documentation

• OPNFV Release R2