Differences

This shows you the differences between two versions of the page.

--- platform_tooling [2014/10/13 13:00]
Morgan Richomme created
+++ platform_tooling [2014/11/06 05:49] (current)
Wenjing Chu [Scope:]
@@ Line 6: / Line 6: @@
   *
 ==== Project description: ====
-  * Provide high level description of intended project and OPNFV use case and benefit, if needed.
+  * High level description of test bed tooling and infrastructure project. We are to support these goals:
-  * Include any architecture diagrams or specifications, reference to OPNFV requirements list.
+   - provide a documented baseline using release candidates of opnfv hw/sw ingredients
+   - release testing of OPNFV's official releases
+   - provide a "sandbox" for developer testing
+   - bottom-up benchmarking & performance testing, all the software and tooling required
+  * Include any architecture diagrams or specifications
+  - The first diagram shows a high level working model for release testing and how this project may interact with other projects
+{{ :high_level_flow_of_release_testing_diagram.png?600 |}}
 ==== Scope: ====
   * Describe the problem being solved by project
-  * Specify any interface/API specification proposed,
+A well established test bed is a critical resource to achieve opnfv's goals. Among these goals, integration of upstream source distributions, operational tools, other software components, hardware infrastructure, validation tools, well thought out quality assurance, well thought out performance metrics benchmarking,... rely heavily on a well established / well understood test bed.
+Many developers may not have either the resources or the time or the inclination to create and maintain such a test bed on their own. It is a substantial challenge to create such an infrastructure in sizable / realistic scale. The well designed / well documented "sandbox" environment can improve developer efficiency and accelerate opnfv's project progress. This benefit can be extended most of the projects.
+Benchmarking a network function can be a complex undertaking with many variables. This project focuses on a bottom-up approach and will likely meet the need of another project that focuses on specific ETSI defined use cases from top down.
+Test bed infrastructure and tooling project extends to all the SW and HW components needed to establish and operate such a test bed. It should:
+   - support both micro/macro level testing, virtual infrastructure level, or VNFC or VNF level testing
+   - can host multi-origin/multi-vendor VNFs, i.e. compositions of vnfc's
+   - support sanity, regression, functional, and performance/benchmark testing
+   - covers both hw and sw components
+The test bed will support the following functionality:
+  - Access tools: ID/authentication, onboarding, terms of use
+  - Access tools: Web access
+  - Access tools: Console access
+  - Physical access
+  - Access tools: Reservation / multi-user/tenant
+  - On-boarding of vnf/vnfc tools
+  - Test automation tools
+  - Devops tools
+  - Diagnostic, logging, monitoring tools
+  - Test load generators, benchmark tools: sw, hw, open source, commercial
+  - Test VNFs, common target or auxiliary VNFs - these VNFCs or VNFs are needed for testing, benchmarking, and for helping developers building their own test scenarios
+  - A standard set of bottom-up test cases
+  - inter-testbed connectivity emulation, if needed
+  - stand-alone nfvc connectivity emulation, if needed
+  - documentation, api reference, test reports
+  * Specify any interface/API specification proposed
+The following diagram shows the high level reference of the test beds and view of the interfaces:
+{{ :high_level_interface_diagram.png?600 |}}
+In high level, the interfaces are:
+  * The interface exposed to an individual developer. This interface is a "user interface".
+  * The interface exposed to another test bed for virtual interconnect.
+  * The interface exposed to an emulation test bed, for instance software in a laptop, for virtual interconnect.
+For the first release, may be only the "user interface" is required.
   * Specify testing and integration:
+Release candidates are delivered from CI project. We need to work out a release candidate cadence in practice to (i) have stability of the test bed for developers who rely on a stable environment, and (ii) meet the need of testing the latest release candidate. A suitable cadence needs to be worked out, or a rolling scheme.
+More complex NFV use cases may be built on the top of this test bed platform. Those use cases may have separate projects / subgroups dedicated to each of them. Additional tooling may be required for specific use cases. But commonly shared test tooling should all be included in this project.
+Some of the testing and benchmarking may be significantly expedited or improved if we can leverage commercial products, either SW and HW. In these cases, some software tools may be required to make the resource available for shared use.
   * Debugging and Tracing
+Debugging and tracing are part of main deliverable for all in scope testing/benchmarking.
   * Unit/Integration Test plans
+Test plans of platform level and tool level instrumentation will be covered in this project. It is assumed that a sister NFV use case project(s) will produce higher level use cases in a top-down approach.
   * Client tools developed for status shows etc.
+Covered in this project includes tools that may go to a developer's client, e.g. for emulation of certain network connectivity. Most of other "client" tools may be supported with web UI.
   * Identity a list of features and functionality will be developed.
+See above in "Scope" section.
   * Identify what is in or out of scope. So during the development phase, it helps reduce discussion.
+In scope:
+  - Bottom-up test cases that validate the functions of a release
+  - Bottom-up benchmark / performance tests
+  - VNF sw to support the above goals
+  - Automation tools to support the above
+  - Software and hardware tools that support developer sandbox
+  - VNF, devops, etc. sw to support the sandbox
+  - Shared common tools
+Out of scope:
+  - Complex NFV use cases
   * Describe how the project is extensible in future
+The project may be expanded in several dimensions: Just some examples.
+  - Additional / different hardware flavors or designs or models
+  - Additional software variations as the overall opnfv support expands, e.g. Linux distribution, OpenStack distributions, Hypervisor / Containers, etc.
+  - Each release of opnfv will bring in new functions
+  - New methods of interacting with the test beds
+  - New instances of test beds
+  - New tools and test cases / use cases that the test bed can support
 ==== Testability: ''(optional, Project Categories: Integration & Testing)'' ====
@@ Line 41: / Line 135: @@
 ==== Committers and Contributors: ====
-  * Name of and affiliation of the maintainer: W.Chu (Dell)
+  * Name of and affiliation of the maintainer: Wenjing Chu (Dell)
   * Names and affiliations of the committers
   * Any other contributors
@@ Line 53: / Line 147: @@
   * When is the first release planned?
-  * Will this align with the current release cadence
+This will be part of the first release.
+  * Will this align with the current release cadence?
+Yes.

Wiki

User Tools

Site Tools

Differences

Page Tools