User Tools

Site Tools


opnfv_functional_testing

This is an old revision of the document!


Project: OPNFV - Base system functionality testing (Functest)

  • Project Name: Functest
  • Project Repository: functest
  • Project Category: Integration & Testing
  • Project IRC chan: #opnfv-testperf

Project description

Project “OPNFV – Base system functionality testing” will provide comprehensive testing methodology, test suites and test cases to test and verify OPNFV Platform functionality that covers the VIM and NFVI components.

This project uses a "top-down" approach that will start with chosen ETSI NFV use-case/s and open source VNFs for the functional testing. The approach taken will be to

  • break down the use-case into simple operations and functions required.
  • specify necessary network topologies
  • develop necessary test traffic
    • Ideally VNFs will be Open Source however proprietary VNFs may also be used as needed.

This project will develop test suites that cover detailed functional test cases, test methodologies and platform configurations which will be documented and maintained in a repository for use by other OPNFV testing projects and the community in general. Developing test suites will also help lay the foundation for a test automation framework that in future can be used by the continuation integration (CI) project (Octopus). We envisage that certain VNF deployment use-cases could be automatically tested as an optional step of the CI process.

The project targets testing of the OPNFV platform in a hosted test-bed environment (i.e. using the OPNFV test labs world wide). It will leverage output of the "BGS" project.

The key objectives are:

  • Define tooling for tests
  • Define test suites (SLA)
  • Installation and configuration of the tools
  • Automate test with CI

A dedicated page for all the testing projects (Functest, Yardstick, Pharos, …) is available here.

Scope

“OPNFV – Base system functionality testing” will deliver a functional testing framework along with a set of test suites and test cases to test and verify the functionality OPNFV platform. The testing framework (tools, test-cases, etc.) are also intended to be used by the CI framework for the purpose of qualifying the OPNFV platform on bare metal servers. In this context, OPNFV Tester will use open source VNF components. Functional testing includes

  • Testing the basic VIM functionality that includes tenant, user CRUD operations, VNF Image CRUD operations etc.
  • Testing the VIM functionality to support VNF operations (create, modify, grow, shrink, destroy)
  • Testing the VIM functionality to support basic VNF network connectivity
  • Testing the inter working between the VIM and the SDN controller.
  • Testing the NFVI functionality as a black box to ensure that it meets the VIM requirements.

The project requires the following components:

  • OPNFV Lab setup with complete access to a set of Bare metal servers for Controller and Compute nodes (as defined by BGS project (OPNFV stack) and Pharos project (hardware)); associated switches and routers.
  • OPNFV platform software bundle from the repository that includes several upstream software components.
  • OPNFV "Bootstart Get Started!" software bundles that includes the installer

Tooling

Functional tests shall be

  • independent from the installer (Fuel, Foreman/Puppet, Juju,..)
  • automated and integrated in CI

Functest versus Release

Testcases

Testcase Availability Comment
vPing Arno basic hello world example.
ODL Arno Using robotframework, 3 pending errors due to ODL bug in Arno.
Tempest Arno Smoke suite considered. Still troubleshooting to be done.
Rally Arno based on default Rally scenario. Still troubleshooting to be done.
vIMS Brahmaputra candidate testcase using Cloudify for VNF management. pre-study
ONOS Brahmaputra candidate using teston tool
SFC Brahmaputra candidate
BGPVPN Brahmaputra candidate
Policy Brahmaputra candidate
vPE ? pre-study
vPE perfo ? pre-study
vEPC ?
vCDN ?

Study on VNF deployment

Tips

Git

Dealing with Branches

Best practice ⇒ Cherry Pick from Master to Stable useful patches as described in Stable Branch

If you forgot that (it may happen and we experienced that for SR1) as Master and stable may be considered as the same, you may have to merge Master on Stable…and then winter is coming…so a procedure that works:

 * git clone ssh://<Your_ID_assuming_you_have_the_merge_rights>@gerrit.opnfv.org:29418/functest functest-clean
 * git checkout stable/arno
 * git merge master
 * git push origin HEAD:refs/for/stable/arno

Dependencies

  • The project is a contributor to project "Pharos", “Octopus” and "Releng".
  • The project leverages "Boot Strap/Get Started"/ "Genesis" and the installer projects
  • The project is part of the test projects (Yardstick, Qtip, Storeperf, ODL, ..

Key Project Facts

This URL is not allowed for scraping
  • Additional Contributors
    • Frank Brockners (fbrockne@cisco.com)
    • Sajeev Manikkoth (sajeevmanikkoth@gmail.com)
    • Sean Chen (s.chen@huawei.com)
    • Rajeev Seth (rseth@sonusnet.com)
    • Kevin Riley (kriley@sonusnet.com)
    • Justin Hart (jhart@sonusnet.com)

Want to Join: know more on Functest

Planned deliverables

Project deliverable: The project delivers the following components:

  • Documentation of Test Suites and test cases that cover the OPNFV platform functionality testing including pass/fail criteria.
  • Test software and scripts for testing OPNFV Platform functionality that is essentially broken down into two main suites as follows:
    • OPNFV Platform Smoke test suite: A sanity testing suite for basic verification of the platform.
    • OPNFV Platform Regression test suite: A comprehensive collection of detailed test suites.
  • Automated Test Framework based on Tempest/Robot/Jenkins, with necessary scripts and tools to automatically test and verify OPNFV functionality

Proposed Release Schedule

OPNFV release #1.

opnfv_functional_testing.1444133789.txt.gz · Last modified: 2015/10/06 12:16 by Morgan Richomme