nfv-kvm-tuning
Differences
This shows you the differences between two versions of the page.
| Both sides previous revision Previous revision Next revision | Previous revision Last revision Both sides next revision | ||
|
nfv-kvm-tuning [2016/01/04 06:37] Chao Peng [Platform Configuration] |
nfv-kvm-tuning [2016/01/04 06:42] Chao Peng [Base Platform and Environment] |
||
|---|---|---|---|
| Line 6: | Line 6: | ||
| * Software: Based on Real-Time Linux https://rt.wiki.kernel.org/index.php/Main_Page. | * Software: Based on Real-Time Linux https://rt.wiki.kernel.org/index.php/Main_Page. | ||
| - | Please refer to [[nfv-kvm-test|kvmfornfv test]] for details. | + | Please refer to [[nfv-kvm-test]] for details. |
| ====== Configuration ====== | ====== Configuration ====== | ||
| Line 31: | Line 31: | ||
| * __Memory allocation__: Memory shoud be reserved for realtime application and usually hugepage should be used to reduce page faut/TLB miss. | * __Memory allocation__: Memory shoud be reserved for realtime application and usually hugepage should be used to reduce page faut/TLB miss. | ||
| - | * __IRQ affinity__: All the non-realtime IRQs should affinitized to non realtime CPUs to reduce the impact on realtime CPUs. Some OS distribution contains irqbalance deamon which balences the IRQs among all the cores dynamically. It should be disabled as well. | + | * __IRQ affinity__: All the non-realtime IRQs should affinitized to non realtime CPUs to reduce the impact on realtime CPUs. Some OS distributions contain irqbalance deamon which balences the IRQs among all the cores dynamically. It should be disabled as well. |
| - | * __Device assignment for VM__: If device is used in a VM, then device passthru is desirable. In this case, IOMMU should be used. | + | * __Device assignment for VM__: If device is used in a VM, then device passthru is desirable. In this case, IOMMU should be enabled. |
| - | * __Tickless__: Frequent tick cause latency. CONFIG_NOHZ_FULL should be enabled in linux kernel.With CONFIG_NOHZ_FULL, the physical CPU will trigger much less tick timer interrupt (currently, 1 tick per second). This will reduce the impact to the VNF because each host timer interrupt triggers VM exit from guest to host and cause performance/latency impact. | + | * __Tickless__: Frequent tick cause latency. CONFIG_NOHZ_FULL should be enabled in linux kernel. With CONFIG_NOHZ_FULL, the physical CPU will trigger much less tick timer interrupt(currently, 1 tick per second). This can reduce latency because each host timer interrupt triggers VM exit from guest to host and cause performance/latency impact. |
| - | * __TSC__: Mark TSC clock source as reliable. A TSC clock source that is thought as unreliable will cause kernel to continuous to enable clock source watchdog to check if TSC frequency is still correct. On latest Intel platform with Constant TSC/Invariant TSC/Synchronized TSC, the TSC is reliable already hence the watchdog is useless but cause latency. | + | * __TSC__: Mark TSC clock source as reliable. A TSC clock source that is thought as unreliable causes kernel to continuous to enable clock source watchdog to check if TSC frequency is still correct. On latest Intel platform with Constant TSC/Invariant TSC/Synchronized TSC, the TSC is reliable already hence the watchdog is useless but cause latency. |
| * __Idle__: The poll option forced a polling idle loop that can slightly improve the performance of waking up an idle CPU. | * __Idle__: The poll option forced a polling idle loop that can slightly improve the performance of waking up an idle CPU. | ||
| Line 43: | Line 43: | ||
| * __RCU_NOCB__: RCU is a kernel synchronization mechanism. Refer to http://lxr.free-electrons.com/source/Documentation/RCU/whatisRCU.txt for more information. With RCU_NOCB, the impact from RCU to the VNF will be reduced. | * __RCU_NOCB__: RCU is a kernel synchronization mechanism. Refer to http://lxr.free-electrons.com/source/Documentation/RCU/whatisRCU.txt for more information. With RCU_NOCB, the impact from RCU to the VNF will be reduced. | ||
| - | * __Disable the RT throttling__: RT Throttling is a Linux kernel mechanism that occurs when a process or thread uses 100% of the core, leaving no resources for the Linux scheduler to execute the kernel/housekeeping tasks. RT Throttling increases the forwarding hence should be disabled. | + | * __Disable the RT throttling__: RT Throttling is a Linux kernel mechanism that occurs when a process or thread uses 100% of the core, leaving no resources for the Linux scheduler to execute the kernel/housekeeping tasks. RT Throttling increases the latency so should be disabled. |
| - | * __NUMA configuration__: To achive the best latency. CPU/Memory and device allocated for realtime application/VM should be in the same NUMA node. | + | * __NUMA configuration__: To achieve the best latency. CPU/Memory and device allocated for realtime application/VM should be in the same NUMA node. |
nfv-kvm-tuning.txt ยท Last modified: 2016/01/13 22:46 by Jiang, Yunhong
Page Tools
Except where otherwise noted, content on this wiki is licensed under the following license: Creative Commons Attribution 4.0 International License
