This is an old revision of the document!
Configuration
A right configuration is critical for improving the NFV performance/latency. Even on the same codebase, different configrations can make completely different performance/latency result.
There are many combinations of configurations, from hardware configuration to Operating System configuration and application level configuration. And there is no one simple configuration that works for every case. To tune a specific scenario, it's important to know the behaviors of different configurations and their impact.
Some hardware features can be configured through firmware interface(e.g. BIOS) but others may not be configurable (e.g. SMI on most platforms).
Power management: Most power management related features save power on the expensive of latency. These features includes: Intel®Turbo Boost Technology, Enhanced Intel®SpeedStep, Processor C state and P state.Normarlly they should be disabled. But depending on the real-time application design and latency requirements,there might be some featues can be enabled if the impact on deterministic execution of workload is small.
Hyper-Threading: The logic cores that share resource with other logic cores can introduce latency so the feature is recommended to disable for realtime case.
Legacy USB Support/Port 60/64 Emulation: These features involve some emulation in firmware and can introduce randome latency. It is recommended to disable.
Operating System Configuration
IRQ affinity: All the non-realtime IRQs should affinitized to non realtime CPUs to reduce the impact on realtime CPUs. Some
OS distribution contains irqbalance deamon which balences the IRQs among all the cores dynamically. It should be disabled as well.
Tickless: Frequent tick cause latency. CONFIG_NOHZ_FULL should be enabled in linux kernel.With CONFIG_NOHZ_FULL, the physical CPU will trigger much less tick timer interrupt (currently, 1 tick per second). This will reduce the impact to the VNF because each host timer interrupt triggers VM exit from guest to host and cause performance/latency impact.
TSC: Mark TSC clock source as reliable. A TSC clock source that is thought as unreliable will cause kernel to continuous to enable clock source watchdog to check if TSC frequency is still correct. On latest Intel platform with Constant TSC/Invariant TSC/Synchronized TSC, the TSC is reliable already hence the watchdog is useless but cause latency.