This is an old revision of the document!
This page discusses the VSPERF Test methodology in detail.
WORK IN PROGRESS - DO NOT PANIC UNTIL INSTRUCTED TO DO SO BY THE CAPTAIN OR A MEMBER OF THE CREW
Test Specification Overview
This section provides detailed information on the test specification and approach that is not covered in the test specification itself.
Test Topologies
 |  |  |  |  |  |  |
TOIT High Level Design
This section provides information about the TOIT framework high level design.
Test Automation with VSPERF
This section provides information about automating the TOIT framework using Jenkins.
The Continuous Integration Process for VSPERF is built on the TOIT framework based on the Jenkins automation tool. Many of the process control technologies used by TOIT are migrated to Jenkins with each of the major functions delegated to a Jenkins job and much of the runtime configuration provided by variables passed to the job by Jenkins.
Each run has the concept of a primary job, this uses the Jenkins Rest API to manage other jobs that provide services. Each configured hypervisor has the concept of a sibling hypervisor which is connected to a hardware packet generator or configured with physical NICs logically connected ‘back to back’
Let’s consider the simplest possible example, Point to Point loopback with OVS and the 2544 data set. This will use 4 Jenkins jobs. 1) The VSPERF P2P management job – The driving job for this test. This job normally runs on Jenkins master 2) The VSPERF build job – This job compiles the VSPERF tree and assembles any artifacts such as packages, drivers to make them available for downstream jobs. The ‘checkinstall’ tool is used to capture any ‘make install’ operations 3) The P2P Switch job is responsible for switch configuration and execution management 4) The Packet Generator VNF Job – Responsible for providing packet generation service 5) The Reporting Job – Responsible for process test artifacts to produce a results document
Now let’s take a look at the workflow. The VSPERF P2P job first launches the build job and monitors its progress using the Jenkins REST API. This job is the same for all VSPERF automation. Once the build has completely successfully the artifacts are archived and the build job exits.
Once the P2P job detects that the build has completed successfully it then launches the P2P Switch job where the real work is done. Based on the build parameters the P2P Switch job first installs the switch under test and does any preliminary configuration. Once the switch is configured the P2P Switch uses the Jenkins REST API to launch the Packet Generator VNF job in a virtual machine on its ‘sibling’ hypervisor by passing this ‘Node’ parameter to the PGVNF job.
Based on the ‘Node’ parameter Jenkins will launch the PG VNF on the sibling node along with any additional parameters such as data type to generate, test duration or other tunable parameters. The Jenkins framework will (optionally) launch the PGVNF VM on demand or may leverage a predefined slave. In either case The PGVNF job is launched as an independent VM and uses the TOIT API combined with build parameters such as the packet generator type to initiate the packet generation process. In the normal case this will run to completion. At the conclusion of the job the PGVNF collects the raw data for reporting which is preserved as a build artifact. In the event of an error the PGVNF job records the error along with any collectible data.
Once the job successfully completes the P2P Switch job is responsible for tear down and clean up of the switch environment such that any other job can reliably run on the same platform. The P2P switch monitors the status of the PGVNF job. Once it detects a successful completion it gathers the reporting artifacts from both the P2P Switch and PGVNF jobs ( if any ). It then launches the Reporting job which massages the input data to create text, web or other reports.
