wiki:ltd_test_spec_overview
Differences
This shows you the differences between two versions of the page.
| Both sides previous revision Previous revision Next revision | Previous revision | ||
|
wiki:ltd_test_spec_overview [2015/09/17 20:32] Al Morton [List of Tests and Descriptions] |
wiki:ltd_test_spec_overview [2015/12/15 23:04] (current) Al Morton [LTD Test Spec Overview] |
||
|---|---|---|---|
| Line 1: | Line 1: | ||
| ====== LTD Test Spec Overview ====== | ====== LTD Test Spec Overview ====== | ||
| - | (This page is currently Under Construction) | + | (This page is currently Under Construction, but now describes all tests as of Sept 17, 2015) |
| - | The Level Test Design (LTD) Specification is one of the products of the OPNFV vsperf project. This page provides an overview of the LTD spec in several forms. | + | The Level Test Design (LTD) Specification is one of the products of the OPNFV vsperf project. The LTD spec is formatted in ReStructured Text (RST), and presented in HTML form with the project [[http://artifacts.opnfv.org/vswitchperf/docs/test_spec/vswitchperf_ltd.html |Artifacts]]. This wiki page provides an overview of the LTD spec in several forms: |
| - | - The LTD Spec and its goals are Summarized in a 20 page [[https://tools.ietf.org/html/draft-vsperf-bmwg-vswitch-opnfv-00|Internet Draft ]] | + | - The LTD Spec and its goals are Summarized in a 21 page [[https://tools.ietf.org/html/draft-vsperf-bmwg-vswitch-opnfv-01|Internet Draft ]] |
| - | - Short descriptions of each of the tests in the LTD Spec (tests developed since July need to be added) | + | - Short descriptions of each of the tests in the LTD Spec (below) |
| - | - Example Device Under Test (DUT) Set-ups, called Deployment Scenarios (to be added) | + | - Example Device Under Test (DUT) Set-ups, called Deployment Scenarios at [[https://wiki.opnfv.org/vsperf/test_methodology|Test Methodology ]] page. |
| + | |||
| + | The approach VSPERF has adopted is to take existing tests that are relevant to performance benchmarking of physical switches, and applying them to benchmarking virtual switches (this is to allow for a fair comparison with their physical counterparts). Once this is done, tests that are relevant to benchmarking virtual switches only, will be developed. | ||
| ===== List of Tests and Descriptions ===== | ===== List of Tests and Descriptions ===== | ||
| Line 12: | Line 14: | ||
| === Speed of Activation === | === Speed of Activation === | ||
| - | o Throughput.RFC2889.AddressLearningRate | + | o Activation.RFC2889.AddressLearningRate |
| This test measures the address learning rate of the DUT, in accordance with [[https://tools.ietf.org/html/rfc2889#section-5.8 |section 5.8 of RFC 2889]]. A minimum of 3 DUT ports are required for this test, a learning port where frames first arrive at the DUT, a test port where frames are forwarded and returned to the source, and a monitoring port where the test device listens for flooded or miss-forwarded frames. | This test measures the address learning rate of the DUT, in accordance with [[https://tools.ietf.org/html/rfc2889#section-5.8 |section 5.8 of RFC 2889]]. A minimum of 3 DUT ports are required for this test, a learning port where frames first arrive at the DUT, a test port where frames are forwarded and returned to the source, and a monitoring port where the test device listens for flooded or miss-forwarded frames. | ||
| Line 45: | Line 47: | ||
| === Scale of Activation === | === Scale of Activation === | ||
| - | o Throughput.RFC2889.AddressCachingCapacity | + | o Activation.RFC2889.AddressCachingCapacity |
| This test measures the address cache capacity of the DUT, in accordance with [[https://tools.ietf.org/html/rfc2889#section-5.7 |section 5.7 of RFC 2889]]. A minimum of 3 DUT ports are required for this test, a learning port where frames first arrive at the DUT, a test port where frames are forwarded and returned to the source, and a monitoring port where the test device listens for flooded or miss-forwarded frames. | This test measures the address cache capacity of the DUT, in accordance with [[https://tools.ietf.org/html/rfc2889#section-5.7 |section 5.7 of RFC 2889]]. A minimum of 3 DUT ports are required for this test, a learning port where frames first arrive at the DUT, a test port where frames are forwarded and returned to the source, and a monitoring port where the test device listens for flooded or miss-forwarded frames. | ||
| Line 64: | Line 66: | ||
| Although **Throughput.RFC2544.PacketLossRatio** is not a prerequisite test, the results may provide a baseline for unmeasurable affect of a CPU intensive application. | Although **Throughput.RFC2544.PacketLossRatio** is not a prerequisite test, the results may provide a baseline for unmeasurable affect of a CPU intensive application. | ||
| - | o Throughput.RFC2544.PacketLossRateFrameModification | + | o Throughput.RFC2544.PacketLossRatioFrameModification |
| This test assesses the [[https://tools.ietf.org/html/rfc2544#section-26.1 | RFC 2544]] Throughput level with additional processes of packet inspection and modification. The Recommended modification is to change or add a VLAN tag, but many more possibilities exist. | This test assesses the [[https://tools.ietf.org/html/rfc2544#section-26.1 | RFC 2544]] Throughput level with additional processes of packet inspection and modification. The Recommended modification is to change or add a VLAN tag, but many more possibilities exist. | ||
| Line 99: | Line 101: | ||
| Although [[https://tools.ietf.org/html/rfc2889#section-5.9 |Section 5.9 of RFC 2889]] indicates this test is a pass or fail outcome, it is appropriate to report the offered load of correct frames and the resulting transmitted frame rate. | Although [[https://tools.ietf.org/html/rfc2889#section-5.9 |Section 5.9 of RFC 2889]] indicates this test is a pass or fail outcome, it is appropriate to report the offered load of correct frames and the resulting transmitted frame rate. | ||
| + | |||
| + | o Throughput.RFC2544.Profile | ||
| + | |||
| + | This is a test to characterize the loss ratio and latency in the overload region, beyond the [[https://tools.ietf.org/html/rfc2544#section-26.1 | RFC 2544]] Throughput level (with 0% loss ratio). The offered load to achieve the Maximum Forwarding Rate in [[https://tools.ietf.org/html/rfc2889#section-5.1 |Section 5.1 of RFC 2889]] testing prescribes the maximum load for this test. | ||
| + | |||
| + | Increased traffic expressed as a percentage of the difference between the Throughput level and the Maximum Offered Load (a physical line rate limitation) is applied to the DUT, in values of -50%, -10%, 0%, +10% and +50%. The Forwarding Rate (frames and Mbps), Latency, CPU & Memory Utilization, and any failures observed should be recorded for all frame sizes tested. | ||
| + | |||
| === Reliability of Operation === | === Reliability of Operation === | ||
| Line 112: | Line 121: | ||
| o PacketDelayVariation.RFC3393.Soak | o PacketDelayVariation.RFC3393.Soak | ||
| + | This is the long duration, or "Soak" test to assess the stability of delay distributions measured in the **Throughput.RFC2544.PacketLossRatio** test. Ideally, the Soak test duration is 24 hours. The [[https://tools.ietf.org/html/rfc5481 | RFC 5481]] PDV form of delay variation is measured on the traffic, using the 99th percentile, for each 60 second interval during the test. This extended period test will determine of there are any long delays and their frequency of occurrence. | ||
| + | |||
| + | The prerequisite test is the **Throughput.RFC2544.PacketLossRatio**, which sets the Throughput level for each frame size. | ||
| === Scalability of Operation === | === Scalability of Operation === | ||
| o Scalability.RFC2544.0PacketLoss | o Scalability.RFC2544.0PacketLoss | ||
| + | |||
| + | This test assesses the [[https://tools.ietf.org/html/rfc2544#section-26.1 | RFC 2544]] Throughput level with a variable number of flows present in the offered load, as recommended in [[https://tools.ietf.org/html/rfc2544#section-12 | section 12 of RFC 2544]], where changing the destination address constitutes a new flow. The number of flows range from 1000 to the Maximum number of supported flows. | ||
| o MemoryBandwidth.RFC2544.0PacketLoss.Scalability | o MemoryBandwidth.RFC2544.0PacketLoss.Scalability | ||
| + | |||
| + | This test assesses the [[https://tools.ietf.org/html/rfc2544#section-26.1 | RFC 2544]] Throughput level while a memory intensive application is running on the same DUT as the vswitch. The test will determine if stressful memory and cache activity, with corresponding memory bus activity, affects performance in the data-path. The test description provides specific requirements for the memory intensive application, including read:write ratio, NUMA architecture considerations, local/remote memory considerations, etc. | ||
| + | |||
| + | Although **Throughput.RFC2544.PacketLossRatio** is not a prerequisite test, the results may provide a baseline for unmeasurable affect of a memory/cache intensive application. | ||
wiki/ltd_test_spec_overview.1442521957.txt.gz · Last modified: 2015/09/17 20:32 by Al Morton
Page Tools
Except where otherwise noted, content on this wiki is licensed under the following license: Creative Commons Attribution 4.0 International License
