project_proposals:yardstick
Differences
This shows you the differences between two versions of the page.
| Next revision | Previous revision | ||
|
project_proposals:yardstick [2015/03/13 10:06] Christopher Price created |
project_proposals:yardstick [2015/04/29 07:56] (current) Ana Cunha Added committer |
||
|---|---|---|---|
| Line 1: | Line 1: | ||
| ==== Project Name: ==== | ==== Project Name: ==== | ||
| - | * Proposed name for the project: ''Yard Stick'' | + | * Proposed name for the project: ''Yardstick'' |
| * Proposed name for the repository: ''yardstick'' | * Proposed name for the repository: ''yardstick'' | ||
| - | * Project Categories: Choose **one** category from the following 4 options: | + | * Project Categories: |
| * Integration & Testing | * Integration & Testing | ||
| ==== Project description: ==== | ==== Project description: ==== | ||
| - | * Provide high level description of intended project and OPNFV use case and benefit, if needed. | + | * This project proposes a test framework, including test cases and test stimuli, to verify the infrastructure compliance when running VNF applications. |
| - | * Include any architecture diagrams or specifications, reference to OPNFV requirements list. | + | |
| ==== Scope: ==== | ==== Scope: ==== | ||
| - | * Describe the problem being solved by project | + | NFV Use Cases described in ETSI GS NFV 001 show a large variety of applications, each defining specific requirements and complex configuration on the underlying infrastructure and test tools. Instead of using one test-VNF application, this project aims at decomposing typical VNF work-load performance metrics into a number of characteristics/performance vectors, which each of them can be represented by distinct test-cases. This approach gives traceable results, thereby providing good fault isolation and flexibility in adding new test cases when a new challenge is identified. |
| - | * Specify any interface/API specification proposed, | + | |
| - | * Specify testing and integration: | + | |
| - | * Debugging and Tracing | + | |
| - | * Unit/Integration Test plans | + | |
| - | * Client tools developed for status shows etc. | + | |
| - | * Identity a list of features and functionality will be developed. | + | |
| - | * Identify what is in or out of scope. So during the development phase, it helps reduce discussion. | + | |
| - | * Describe how the project is extensible in future | + | |
| - | ==== Testability: ''(optional, Project Categories: Integration & Testing)'' ==== | + | The metrics shall be divided into categories, such as: |
| - | * Specify testing and integration like interoperability, scalability, high availablity | + | * Compute |
| - | * What QA and test resources will be available? | + | * Storage |
| + | * Networking | ||
| - | ==== Documentation: ''(optional, Project Categories: Documention)'' ==== | + | The test cases might be combined and run in paralel, enabling: |
| - | * API Docs | + | * verification of simple as well as complex scenarios |
| - | * Functional block description | + | * verification of resource-demanding requirements such as simulating millions of flows, different traffic mixes, burstiness, fault injection |
| + | |||
| + | The proposed methodology includes: | ||
| + | |||
| + | * simple test programs, which breaks down VNF application requirements into several simple test cases | ||
| + | * flexible test framework, allowing introduction of new test cases when needed/new challenge discovered | ||
| + | * good fault isolation, enabling early fault detection | ||
| + | * test stimuli, to enable infrastructure testing such as infrastructure upgrade, infrastructure recovery | ||
| + | * test framework based upon open APIs (OpenStack/Heat) for easy deployment | ||
| + | |||
| + | This project will develop additional functionality to the base system testing covered by the project "Functest", such as metrics, parallel testing, fault injection, multiple topologies, scenario testing, enabling verification of more complex test cases. | ||
| + | |||
| + | This project proposes to contribute a first implementation for evaluation, which: | ||
| + | |||
| + | * uses standard OpenStack APIs | ||
| + | * written in Python | ||
| + | * webDAV file system | ||
| + | * supports affinity/anti-affinity | ||
| + | * orchestration using Heat templates | ||
| + | * logging using the facility provided with Python | ||
| + | * documentation | ||
| + | * test case examples | ||
| + | * can be run as stand-alone | ||
| + | |||
| + | Candidates for functionality to be developed: | ||
| + | |||
| + | * Result presentation | ||
| + | * File System | ||
| + | * Simplified API | ||
| + | * Additional Networking topologies | ||
| + | * Additional test cases and examples | ||
| + | * Conformance testing | ||
| + | |||
| + | Presentation | ||
| + | |||
| + | {{:project_proposals:yardstick_proposal.pptx|}} | ||
| + | |||
| + | |||
| + | ==== Testability: ==== | ||
| + | |||
| + | * To be integrated in OPNFV Continuous Integration testing. | ||
| ==== Dependencies: ==== | ==== Dependencies: ==== | ||
| - | * Identify similar projects is underway or being proposed in OPNFV or upstream project | + | * The test framework is hardware, VNF application, operator and vendor independent. |
| - | * Identify any open source upstream projects and release timeline. | + | * Current limitation: OpenStack. |
| - | * Identify any specific development be staged with respect to the upstream project and releases. | + | * Yardstick will consider outputs of the ETSI ISG NFV about testing an NFV environment. Only public documents can be used, so Yardstick might need to adapt the timeplan for the deliverables where output from ongoing work, e.g. from TST working group needs to be considered, accordingly. |
| - | * Are there any external fora or standard development organization dependencies. If possible, list and informative and normative reference specifications. | + | * Since verification as done in Yardstick has to prepare the ground for any certification activity, the Yardstick project is dependent on the Certification & Compliance Committee and will put those issues on high priority that are needed for any certification activities that will be needed for certification or compliance if there will be a decision to engage in this. |
| - | * If project is an integration and test, identify hardware dependency. | + | |
| ==== Committers and Contributors: ==== | ==== Committers and Contributors: ==== | ||
| + | * Name and affiliation of the project leader: | ||
| + | * Ana Cunha (Ericsson): ana.cunha@ericsson.com | ||
| * Names and affiliations of the committers | * Names and affiliations of the committers | ||
| - | * Names and affiliations of any other contributors | + | * Ana Cunha (Ericsson): ana.cunha@ericsson.com |
| + | * Leif Johansson (Ericsson): leif.y.johansson@ericsson.com | ||
| + | * Hans Feldt (Ericsson): hans.feldt@ericsson.com | ||
| + | * Jörgen Karlsson (Ericsson): jorgen.w.karlsson@ericsson.com | ||
| + | * Houjingwen (Huawei): houjingwen@huawei.com | ||
| + | * Trevor Cooper (Intel): trevor.cooper@intel.com | ||
| + | * Wenjing Chu (DELL): Wenjing_Chu@DELL.com | ||
| + | * Names and affiliations of other contributors | ||
| + | * Morgan Richomme (Orange): morgan.richomme@orange.com | ||
| + | * Dr. Rebecca Steinert (SICS): rebste@sics.se | ||
| + | * Fatih Degirmenci (Ericsson): fatih.degirmenci@ericsson.com | ||
| + | * Pierre Lynch (Ixia): PLynch@ixiacom.com | ||
| + | * Julien Zhang (ZTE) :zhang.jun3g@zte.com.cn | ||
| + | |||
| ==== Planned deliverables ==== | ==== Planned deliverables ==== | ||
| - | + | ||
| - | * Described the project release package as OPNFV or open source upstream projects. | + | * Test framework |
| - | * If project deliverables have multiple dependencies across other project categories, described linkage of the deliverables. | + | * Test cases examples |
| + | * Reference documentation | ||
| ==== Proposed Release Schedule: ==== | ==== Proposed Release Schedule: ==== | ||
| - | * When is the first release planned? | + | * OPNFV Release R2 |
| - | * Will this align with the current release cadence | + | |
project_proposals/yardstick.1426241167.txt.gz · Last modified: 2015/03/13 10:06 by Christopher Price
Page Tools
Except where otherwise noted, content on this wiki is licensed under the following license: Creative Commons Attribution 4.0 International License
