project_proposals:yardstick
Differences
This shows you the differences between two versions of the page.
| Both sides previous revision Previous revision Next revision | Previous revision | ||
|
project_proposals:yardstick [2015/03/16 11:37] Ana Cunha |
project_proposals:yardstick [2015/04/29 07:56] (current) Ana Cunha Added committer |
||
|---|---|---|---|
| Line 1: | Line 1: | ||
| ==== Project Name: ==== | ==== Project Name: ==== | ||
| - | * Proposed name for the project: ''Yard Stick'' | + | * Proposed name for the project: ''Yardstick'' |
| * Proposed name for the repository: ''yardstick'' | * Proposed name for the repository: ''yardstick'' | ||
| * Project Categories: | * Project Categories: | ||
| Line 7: | Line 7: | ||
| ==== Project description: ==== | ==== Project description: ==== | ||
| - | * This project proposes a test framework, including test cases, to verify the infrastructure compliance when running VNF applications. | + | * This project proposes a test framework, including test cases and test stimuli, to verify the infrastructure compliance when running VNF applications. |
| ==== Scope: ==== | ==== Scope: ==== | ||
| - | * Challenges when verifying a VNF application: | + | NFV Use Cases described in ETSI GS NFV 001 show a large variety of applications, each defining specific requirements and complex configuration on the underlying infrastructure and test tools. Instead of using one test-VNF application, this project aims at decomposing typical VNF work-load performance metrics into a number of characteristics/performance vectors, which each of them can be represented by distinct test-cases. This approach gives traceable results, thereby providing good fault isolation and flexibility in adding new test cases when a new challenge is identified. |
| - | * VNF applications are very different | + | |
| - | * Complex to configure/handle | + | The metrics shall be divided into categories, such as: |
| - | * Faults within VNF application are very complex to debug/understand | + | |
| - | + | * Compute | |
| - | * Solution: | + | * Storage |
| - | * Simple test programs which breaks down VNF application requirements into several simple test cases | + | * Networking |
| - | * Flexible test framework to introduce new test cases when needed/new problem discovered | + | |
| - | * Good fault isolation to be able to identify/detect faults early | + | The test cases might be combined and run in paralel, enabling: |
| - | * Test framework based upon open APIs (OpenStack/Heat) for easy deployment. | + | |
| + | * verification of simple as well as complex scenarios | ||
| + | * verification of resource-demanding requirements such as simulating millions of flows, different traffic mixes, burstiness, fault injection | ||
| + | |||
| + | The proposed methodology includes: | ||
| + | |||
| + | * simple test programs, which breaks down VNF application requirements into several simple test cases | ||
| + | * flexible test framework, allowing introduction of new test cases when needed/new challenge discovered | ||
| + | * good fault isolation, enabling early fault detection | ||
| + | * test stimuli, to enable infrastructure testing such as infrastructure upgrade, infrastructure recovery | ||
| + | * test framework based upon open APIs (OpenStack/Heat) for easy deployment | ||
| + | |||
| + | This project will develop additional functionality to the base system testing covered by the project "Functest", such as metrics, parallel testing, fault injection, multiple topologies, scenario testing, enabling verification of more complex test cases. | ||
| - | * Proposed implementation of test framework addressing identified test requirements/needs: | + | This project proposes to contribute a first implementation for evaluation, which: |
| - | * Uses standart OpenStack APIs | + | |
| - | * Written in Python | + | |
| - | * WebDAV file system | + | |
| - | * Supports affinity/anti-affinity | + | |
| - | * Orchestration using Heat templates | + | |
| - | * Logging using the facility provided with Python | + | |
| - | * Documentation generated using Sphinx | + | |
| - | * Testing and integration: | + | * uses standard OpenStack APIs |
| - | * The test can be used stand-alone in operators standard test environment | + | * written in Python |
| + | * webDAV file system | ||
| + | * supports affinity/anti-affinity | ||
| + | * orchestration using Heat templates | ||
| + | * logging using the facility provided with Python | ||
| + | * documentation | ||
| + | * test case examples | ||
| + | * can be run as stand-alone | ||
| - | * Candidates for functionality to be developed. | + | Candidates for functionality to be developed: |
| - | * Result presentation | + | |
| - | * Swift implementation | + | * Result presentation |
| - | * Simplified API | + | * File System |
| - | * Additional Networking topologies | + | * Simplified API |
| - | * Additional test cases and examples | + | * Additional Networking topologies |
| - | * Support for different deployment configuration | + | * Additional test cases and examples |
| + | * Conformance testing | ||
| + | |||
| + | Presentation | ||
| + | |||
| + | {{:project_proposals:yardstick_proposal.pptx|}} | ||
| + | | ||
| ==== Testability: ==== | ==== Testability: ==== | ||
| - | * To be integrated in CI testing. | + | * To be integrated in OPNFV Continuous Integration testing. |
| ==== Dependencies: ==== | ==== Dependencies: ==== | ||
| - | * The test framework is hardware, VNF application, operator and vendor independent | + | * The test framework is hardware, VNF application, operator and vendor independent. |
| - | * Current limitation: OpenStack | + | * Current limitation: OpenStack. |
| + | * Yardstick will consider outputs of the ETSI ISG NFV about testing an NFV environment. Only public documents can be used, so Yardstick might need to adapt the timeplan for the deliverables where output from ongoing work, e.g. from TST working group needs to be considered, accordingly. | ||
| + | * Since verification as done in Yardstick has to prepare the ground for any certification activity, the Yardstick project is dependent on the Certification & Compliance Committee and will put those issues on high priority that are needed for any certification activities that will be needed for certification or compliance if there will be a decision to engage in this. | ||
| ==== Committers and Contributors: ==== | ==== Committers and Contributors: ==== | ||
| - | |||
| * Name and affiliation of the project leader: | * Name and affiliation of the project leader: | ||
| - | * Ana Cunha: ana.cunha@ericsson.com | + | * Ana Cunha (Ericsson): ana.cunha@ericsson.com |
| * Names and affiliations of the committers | * Names and affiliations of the committers | ||
| - | * Ana Cunha: ana.cunha@ericsson.com | + | * Ana Cunha (Ericsson): ana.cunha@ericsson.com |
| - | * Leif Johansson: leif.y.johansson@ericsson.com | + | * Leif Johansson (Ericsson): leif.y.johansson@ericsson.com |
| - | * Names and affiliations of any other contributors | + | * Hans Feldt (Ericsson): hans.feldt@ericsson.com |
| - | + | * Jörgen Karlsson (Ericsson): jorgen.w.karlsson@ericsson.com | |
| + | * Houjingwen (Huawei): houjingwen@huawei.com | ||
| + | * Trevor Cooper (Intel): trevor.cooper@intel.com | ||
| + | * Wenjing Chu (DELL): Wenjing_Chu@DELL.com | ||
| + | * Names and affiliations of other contributors | ||
| + | * Morgan Richomme (Orange): morgan.richomme@orange.com | ||
| + | * Dr. Rebecca Steinert (SICS): rebste@sics.se | ||
| + | * Fatih Degirmenci (Ericsson): fatih.degirmenci@ericsson.com | ||
| + | * Pierre Lynch (Ixia): PLynch@ixiacom.com | ||
| + | * Julien Zhang (ZTE) :zhang.jun3g@zte.com.cn | ||
| + | |||
| ==== Planned deliverables ==== | ==== Planned deliverables ==== | ||
| Line 70: | Line 99: | ||
| ==== Proposed Release Schedule: ==== | ==== Proposed Release Schedule: ==== | ||
| - | * Release R2 | + | * OPNFV Release R2 |
project_proposals/yardstick.1426505827.txt.gz · Last modified: 2015/03/16 11:37 by Ana Cunha
Page Tools
Except where otherwise noted, content on this wiki is licensed under the following license: Creative Commons Attribution 4.0 International License
