Qtip project is a "Bottom-Up" approach in characterizing and benchmarking OPNFV Platform. Proper characterization of OPNFV platforms is a critical component to understand how well OPNFV (and by implication upstream components) perform in realistic deployment scenarios, to provide feedback to design teams, to provide platform level performance data to higher layer developers, users, and community at large.
Qtip also aims to build necessary testing and benchmarking tools that will be needed to fulfill its goals, to automate when possible, and to provide these tools and automation software to upper layer and other testing projects in OPNFV.
The overall problem this project tries to solve is the general characterization of an OPNFV platform. It will focus on general performance questions that are common to the platform itself, or applicable to multiple OPNFV use cases. QTIP will provide the capability to quantify a platform's performance behavior in a standardized, rigorous, and open way, and a well-documented methodology to reproduce the results by anyone interested
Main activities include:
In order to achieve the above objectives, we intend to use both open source, and non-open source commercial tools under guidelines developed by the community.
Such sophisticated testing methodology and harness, once developed, will first be applied to quantify a platform's behavior from lower layer going up (Bottom-up Approach). We believe understanding lower layer behavior is a necessary prerequisite step of understanding more complex upper layer behavior with more interconnecting parts. Examples of such platform level sub-systems may include (for illustration only): CPU, Memory, NICs, Storage, Switching, Hypervisors, containers, Host OS, Guest OS, vSwitch/vForwarding, TCP/IP, base OpenStack, base ODL/SDN and so on.
Computing Benchmarks
Running QTIP benchmarks on different machines configured with different components would evaluate the influence of these components on the computing performance. For example, comparing these benchmarks on baremetal machines with different CPUs would help evaluate the performance of the CPU. Same approach can be used to test memory performance. Comparing the performance of these benchmarks on a baremetal machine vs a VM running on the same baremetal machine would help analyze the overhead of the hypervisor.
Some of the computing benchmarks include:
Networking Benchmarks
More details within the coming week.
Storage Benchmarks
These benchmarks can be used to compare storage peformance for disks mounted locally as well on a network. The different storage components within different platforms can be compared using these benchmarks to get quantitative results for storage performance.
Software
Related projects
For Brahamputra release, QTIP would be calculating three indices.
These indices are calculated by comparing QTIP results with the QTIP reference results obtained at Dell's OPNFV Lab's POD 3
Committers:
Contributors: