This is an old revision of the document!
Project Name:
Project description:
Scope:
NFV Use Cases described in ETSI GS NFV 001 show a large variety of applications, each defining specific requirements and complex configuration on the underlying infrastructure and test tools. Instead of using one test-VNF application, this project aims at decomposing typical VNF work-load performance metrics into a number of characteristics/performance vectors, which each of them can be represented by distinct test-cases. This approach gives traceable results, thereby providing good fault isolation and flexibility in adding new test cases when a new challenge is identified.
The metrics shall be divided into categories, such as:
Compute
Storage
Networking
The test cases might be combined and run in paralel, enabling:
verification of simple as well as complex scenarios
verification of resource-demanding requirements such as simulating millions of flows, different traffic mixes, burstiness, fault injection
The proposed methodology includes:
simple test programs, which breaks down VNF application requirements into several simple test cases
flexible test framework, allowing introduction of new test cases when needed/new challenge discovered
good fault isolation, enabling early fault detection
test stimuli, to enable infrastructure testing such as infrastructure upgrade, infrastructure recovery
test framework based upon open APIs (OpenStack/Heat) for easy deployment
This project proposes to contribute a first implementation for evaluation, which:
uses standard OpenStack APIs
written in Python
webDAV file system
supports affinity/anti-affinity
orchestration using Heat templates
logging using the facility provided with Python
documentation
test case examples
can be run as stand-alone
Candidates for functionality to be developed:
Presentation
yardstick_proposal.pptx
Testability:
Dependencies:
The test framework is hardware, VNF application, operator and vendor independent.
Current limitation: OpenStack.
Yardstick will consider outputs of the ETSI ISG NFV about testing an NFV environment. Only public documents can be used, so Yardstick might need to adapt the timeplan for the deliverables where output from ongoing work, e.g. from TST working group needs to be considered, accordingly.
Yardstick will contribute to the project "Functest".
Since verification as done in Yardstick has to prepare the ground for any certification activity, the Yardstick project is dependent on the Certification & Compliance Committee and will put those issues on high priority that are needed for any certification activities that will be needed for certification or compliance if there will be a decision to engage in this.
Committers and Contributors:
Name and affiliation of the project leader:
Names and affiliations of the committers
Ana Cunha (Ericsson): ana.cunha@ericsson.com
Leif Johansson (Ericsson): leif.y.johansson@ericsson.com
Hans Feldt (Ericsson): hans.feldt@ericsson.com
Jörgen Karlsson (Ericsson): jorgen.w.karlsson@ericsson.com
Names and affiliations of other contributors
Morgan Richomme (Orange): morgan.richomme@orange.com
Dr. Rebecca Steinert (SICS): rebste@sics.se
Fatih Degirmenci (Ericsson): fatih.degirmenci@ericsson.com
Pierre Lynch (Ixia): PLynch@ixiacom.com
Planned deliverables
Test framework
Test cases examples
Reference documentation
Proposed Release Schedule: