Methodology
===== From September 23rd, 2015 the information on this page is no longer maintained, refer to Jira links for information about Yardstick planned and current work. ====
 | Looking for Yardstick Status & Test Cases? Yardstick Jira Dashboard Yardstick Kanban board Yardstick Test cases |
|---|
====== Proposed Methodology ======
This page describes the proposed methodology in the Yardstick Project.
The methodology consists in decomposing typical VNF work-load performance metrics into a number of characteristics/performance vectors, which each of them can be represented by distinct test-cases.
The metrics shall be divided into areas:
- Compute
- Storage
- Networking
Add your metric suggestion to the tables below.
Way of Working: Once the team decides to implement a specific metric, a link to the corresponding Jira issue will be added.
Compute Area
| Metric | Tool | Test Case | Notes |
|---|---|---|---|
| <Metric> | <Use this tool to measure> | <test case that deliveres the metric> | <Notes, e.g. Tool already in framework?, Dependencies? > |
| # cores | |||
| # threads | |||
| # virtual machines | |||
| processor utilization (%) | |||
| processing speed (instructions/s) | |||
| memory consumption | |||
| cpu load (max, average, std deviation) | Linux Kernel | https://jira.opnfv.org/browse/YARDSTICK-12 | |
| cache performance | Linux Kernel | https://jira.opnfv.org/browse/YARDSTICK-14 | |
| response time | |||
| throughput | |||
| bandwidth | |||
| availability | |||
| QoS |
Storage Area
| Metric | Tool | Test Case | Notes |
|---|---|---|---|
| <Metric> | <Use this tool to measure> | <test case that deliveres the metric> | <Notes, e.g. Tool already in framework?, Dependencies? > |
| # read/write | |||
| storage size | |||
| capacity allocation (object-based, block-based) | |||
| block size | |||
| usage | |||
| QoS |
Networking Area
| Metric | Tool | Test Case | Notes |
|---|---|---|---|
| <Metric> | <Use this tool to measure> | <test case that deliveres the metric> | <Notes, e.g. Tool already in framework?, Dependencies? > |
| # packets sent | pktgen | https://jira.opnfv.org/browse/YARDSTICK-5 | |
| # packets received | pktgen | https://jira.opnfv.org/browse/YARDSTICK-5 | |
| # packets lost (ppm) | https://jira.opnfv.org/browse/YARDSTICK-5 | ||
| # flows | pktgen | https://jira.opnfv.org/browse/YARDSTICK-5 | |
| latency per packet | |||
| jitter | |||
| network bandwidth | |||
| link availability | |||
| QoS |