nfv-kvm-test
Differences
This shows you the differences between two versions of the page.
| Both sides previous revision Previous revision Next revision | Previous revision Last revision Both sides next revision | ||
|
nfv-kvm-test [2015/11/09 21:21] Jiang, Yunhong [Test Description] |
nfv-kvm-test [2016/01/04 06:17] Chao Peng [Kernel Parameter] |
||
|---|---|---|---|
| Line 126: | Line 126: | ||
| ==== Kernel Parameter ==== | ==== Kernel Parameter ==== | ||
| - | Several kernel parameters are important to achieve real performance for the VNF. | ||
| - | 1. Setup the isolated CPUs | + | __Kernel configuration__: kernel/arch/x86/configs/opnfv.config. |
| - | Isolated CPUs make sure the kernel will not schedule process to the specified isolated CPUs unless requested explicitly. Please refer to http://lxr.free-electrons.com/source/Documentation/kernel-parameters.txt#L1608 for more information. Normally, the physical CPUs planned to host the VNF should be marked as isolated CPUs. | + | |
| - | 2. Setup the NOHZ_FULL | + | __Host kernel boot line example__: isolcpus=11-15,31-35 nohz_full=11-15,31-35 rcu_nocbs=11-15,31-35 iommu=pt intel_iommu=on default_hugepagesz=1G hugepagesz=1G mce=off idle=poll intel_pstate=disable processor.max_cstate=1 pcie_asmp=off tsc=reliable |
| - | NOHZ_FULL is meaningful to reduce the jitter caused by host to the VNF. With NOHZ_FULL, the physical CPU will trigger much less tick timer interrupt (currently, 1 tick per second). This will reduce the impact to the VNF, because each host timer interrupt will trigger VM exit from guest to host and cause performance/latency impact. | + | |
| - | 3. Setup the RCU_NOCB | + | __Guest kernel boot line example__: isolcpus=1 nohz_full=1 rcu_nocbs=1 mce=off idle=poll default_hugepagesz=1G hugepagesz=1G |
| - | RCU is a kernel synchronization mechanism. Refer to http://lxr.free-electrons.com/source/Documentation/RCU/whatisRCU.txt for more information. With RCU_NOCB, the impact from RCU to the VNF will be reduced a lot. | + | |
| - | 4. Mark the tsc as reliable | + | Please refer to [[nfv-kvm-tuning]] for more explanation. |
| - | Please notice that this is a little bit hacky while we are providing a cleaner solution. A TSC clock source that is thought as unreliable will cause kernel to continuous to enable clock source watchdog to check if TSC frequency is still correct. On latest Intel platform with Constant TSC/Invariant TSC/Synchronized TSC, the TSC is reliable already. We are on the process to submit a patch to the kernel. | + | |
| ==== Run-time Environment Setup ==== | ==== Run-time Environment Setup ==== | ||
| Line 163: | Line 158: | ||
| We collected baseline performance data, i.e. the performance data without any enhancement, but with all the system configuration applied already. | We collected baseline performance data, i.e. the performance data without any enhancement, but with all the system configuration applied already. | ||
| + | |||
| ===== Baseline Performance Data ===== | ===== Baseline Performance Data ===== | ||
| + | 1. CyclicTest | ||
| + | <code> | ||
| + | # Min Latencies: 00005 | ||
| + | # Avg Latencies: 00007 | ||
| + | # Max Latencies: 00032 | ||
| + | # Histogram Overflows: 00002 | ||
| + | # Histogram Overflow at cycle number: | ||
| + | # Thread 0: 45337966 50108448 | ||
| + | </code> | ||
| + | |||
| + | 2. Device Interrupt Latency | ||
| + | <code> | ||
| + | Latency is Min: 7.22us Max: 55.28us Avg: 8.93us | ||
| + | Jitter is Min: 963.20us Max: 1040.68us Avg: 1002.62us | ||
| + | </code> | ||
| + | |||
| ===== Latest Performance Data ===== | ===== Latest Performance Data ===== | ||
| + | |||
| + | This is the test result at 11/09/2015. | ||
| + | |||
| + | 1. CyclictTest | ||
| + | <code> | ||
| + | # Total: 086400000 | ||
| + | # Min Latencies: 00006 | ||
| + | # Avg Latencies: 00006 | ||
| + | # Max Latencies: 00011 | ||
| + | # Histogram Overflows: 00000 | ||
| + | # Histogram Overflow at cycle number: | ||
| + | # Thread 0: | ||
| + | </code> | ||
| + | |||
| + | 2. Device Interrupt Latency | ||
| + | <code> | ||
| + | Period is 1000 us | ||
| + | Latency is Min: 4.11us Max: 9.11us Avg: 5.06us | ||
| + | Jitter is Min: 997.09us Max: 1008.23us Avg: 1002.62us | ||
| + | </code> | ||
| + | |||
| + | ===== Latest Live Migration Performance Data ===== | ||
| + | |||
| + | For live migration performance data, please refer to https://etherpad.opnfv.org/p/live_migration_optimization_data for information. | ||
| + | |||
nfv-kvm-test.txt · Last modified: 2016/01/04 06:24 by Chao Peng
Page Tools
Except where otherwise noted, content on this wiki is licensed under the following license: Creative Commons Attribution 4.0 International License
