User Tools

Site Tools


platform_tooling

Differences

This shows you the differences between two versions of the page.

Link to this comparison view

Both sides previous revision Previous revision
Next revision
Previous revision
platform_tooling [2014/10/24 19:08]
Wenjing Chu
platform_tooling [2014/11/06 05:49] (current)
Wenjing Chu [Scope:]
Line 11: Line 11:
    - release testing of OPNFV'​s official releases    - release testing of OPNFV'​s official releases
    - provide a "​sandbox"​ for developer testing    - provide a "​sandbox"​ for developer testing
 +   - bottom-up benchmarking & performance testing, all the software and tooling required
  
  
Line 16: Line 17:
  
   - The first diagram shows a high level working model for release testing and how this project may interact with other projects   - The first diagram shows a high level working model for release testing and how this project may interact with other projects
-{{ :high_level_flow_of_release_testing.png?800 |}}+{{ :high_level_flow_of_release_testing_diagram.png?600 |}}
 ==== Scope: ==== ==== Scope: ====
  
   * Describe the problem being solved by project   * Describe the problem being solved by project
  
-   - support both micro/macro level testing, virtual infrastructure level or VNFC or VNF level testing+A well established test bed is a critical resource to achieve opnfv'​s goals. Among these goals, integration of upstream source distributions,​ operational tools, other software components, hardware infrastructure,​ validation tools, well thought out quality assurance, well thought out performance metrics benchmarking,​... rely heavily on a well established / well understood test bed. 
 + 
 +Many developers may not have either the resources or the time or the inclination to create and maintain such a test bed on their own. It is a substantial challenge to create such an infrastructure in sizable / realistic scale. The well designed / well documented "​sandbox"​ environment can improve developer efficiency and accelerate opnfv'​s project progress. This benefit can be extended most of the projects. 
 + 
 +Benchmarking a network function can be a complex undertaking with many variables. This project focuses on a bottom-up approach and will likely meet the need of another project that focuses on specific ETSI defined use cases from top down. 
 + 
 +Test bed infrastructure and tooling project extends to all the SW and HW components needed to establish and operate such a test bed. It should: 
 + 
 +   - support both micro/macro level testing, virtual infrastructure levelor VNFC or VNF level testing
    - can host multi-origin/​multi-vendor VNFs, i.e. compositions of vnfc's    - can host multi-origin/​multi-vendor VNFs, i.e. compositions of vnfc's
-   - support regression, functional, and performance/​benchmark testing +   - support ​sanity, ​regression, functional, and performance/​benchmark testing 
-   - covers both hw and sw components+   - covers both hw and sw components ​ 
 + 
 +The test bed will support the following functionality:​
  
-The test bed tooling functionality will cover: +  ​- Access tools: ID/​authentication, onboarding, terms of use
-  ​- Access tools: ID/​authentication+
   - Access tools: Web access   - Access tools: Web access
   - Access tools: Console access   - Access tools: Console access
 +  - Physical access
   - Access tools: Reservation / multi-user/​tenant   - Access tools: Reservation / multi-user/​tenant
   - On-boarding of vnf/vnfc tools   - On-boarding of vnf/vnfc tools
   - Test automation tools   - Test automation tools
 +  - Devops tools
   - Diagnostic, logging, monitoring tools   - Diagnostic, logging, monitoring tools
-  - Test load generators, benchmark tools: sw, hw +  - Test load generators, benchmark tools: sw, hw, open source, commercial 
-  - Test VNFs, target or auxiliary ​+  - Test VNFs, common ​target or auxiliary ​VNFs - these VNFCs or VNFs are needed for testing, benchmarking,​ and for helping developers building their own test scenarios 
 +  - A standard set of bottom-up test cases
   - inter-testbed connectivity emulation, if needed   - inter-testbed connectivity emulation, if needed
   - stand-alone nfvc connectivity emulation, if needed   - stand-alone nfvc connectivity emulation, if needed
Line 44: Line 57:
  
 The following diagram shows the high level reference of the test beds and view of the interfaces: The following diagram shows the high level reference of the test beds and view of the interfaces:
-{{ :high_level_interface_view.png?800 |}}+{{ :high_level_interface_diagram.png?600 |}} 
 + 
 +In high level, the interfaces are: 
 +  * The interface exposed to an individual developer. This interface is a "user interface"​. 
 +  * The interface exposed to another test bed for virtual interconnect. 
 +  * The interface exposed to an emulation test bed, for instance software in a laptop, for virtual interconnect. 
 +For the first release, may be only the "user interface"​ is required.
  
   * Specify testing and integration:​   * Specify testing and integration:​
  
-Release candidates are delivered from CI project. We need to work out a release candidate cadence in practice to (i) have stability of the test bed for developers who are rely on a stable environment,​ and (ii) meet the need of testing the latest release candidate. A suitable cadence needs to be worked out, or a rolling scheme.+Release candidates are delivered from CI project. We need to work out a release candidate cadence in practice to (i) have stability of the test bed for developers who rely on a stable environment,​ and (ii) meet the need of testing the latest release candidate. A suitable cadence needs to be worked out, or a rolling scheme
 + 
 +More complex NFV use cases may be built on the top of this test bed platform. Those use cases may have separate projects / subgroups dedicated to each of them. Additional tooling may be required for specific use cases. But commonly shared test tooling should all be included in this project. 
 + 
 +Some of the testing and benchmarking may be significantly expedited or improved if we can leverage commercial products, either SW and HW. In these cases, some software tools may be required to make the resource available for shared use.
  
   * Debugging and Tracing ​   * Debugging and Tracing ​
  
-Debugging and tracing are part of main deliverables ​for all in scope testing.+Debugging and tracing are part of main deliverable ​for all in scope testing/​benchmarking.
  
   * Unit/​Integration Test plans   * Unit/​Integration Test plans
  
-Test plans of platform level and tool level instrumentation will be covered in this project. It is assumed that a sister ​VNF test case project will produce higher level use cases.+Test plans of platform level and tool level instrumentation will be covered in this project. It is assumed that a sister ​NFV use case project(s) will produce higher level use cases in a top-down approach.
  
   * Client tools developed for status shows etc.    * Client tools developed for status shows etc. 
Line 63: Line 86:
  
   * Identity a list of features and functionality will be developed.   * Identity a list of features and functionality will be developed.
 +
 +See above in "​Scope"​ section.
 +
   * Identify what is in or out of scope. So during the development phase, it helps reduce discussion.   * Identify what is in or out of scope. So during the development phase, it helps reduce discussion.
 +
 +In scope:
 +  - Bottom-up test cases that validate the functions of a release
 +  - Bottom-up benchmark / performance tests
 +  - VNF sw to support the above goals
 +  - Automation tools to support the above
 +  - Software and hardware tools that support developer sandbox
 +  - VNF, devops, etc. sw to support the sandbox
 +  - Shared common tools
 +
 +Out of scope:
 +  - Complex NFV use cases
 +
   * Describe how the project is extensible in future   * Describe how the project is extensible in future
 +
 +The project may be expanded in several dimensions: Just some examples.
 +  - Additional / different hardware flavors or designs or models
 +  - Additional software variations as the overall opnfv support expands, e.g. Linux distribution,​ OpenStack distributions,​ Hypervisor / Containers, etc.
 +  - Each release of opnfv will bring in new functions
 +  - New methods of interacting with the test beds
 +  - New instances of test beds
 +  - New tools and test cases / use cases that the test bed can support
 +
 +
  
 ==== Testability:​ ''​(optional,​ Project Categories: Integration & Testing)''​ ==== ==== Testability:​ ''​(optional,​ Project Categories: Integration & Testing)''​ ====
platform_tooling.1414177727.txt.gz · Last modified: 2014/10/24 19:08 by Wenjing Chu