User Tools

Site Tools


vm2vm_mst

Differences

This shows you the differences between two versions of the page.

Link to this comparison view

Both sides previous revision Previous revision
Next revision
Previous revision
Last revision Both sides next revision
vm2vm_mst [2015/09/03 02:25]
Jun Nakajima
vm2vm_mst [2015/09/03 04:33]
Jun Nakajima
Line 6: Line 6:
 ---- ----
  
-VM1, VM2 corresponding ​to the examples in his proposal. ​Basically ​VM1 can express access permission (R/W) to its guest physical address (GPA) space by virtual IOMMU. Typically, IOMMU (e.g.  AMD-Vi and Intel VT-d) uses page tables that convert bus (I/O) address to GPA for a given PCI device to protect the rest of the system from the DMA operations. See [[https://​www.kernel.org/​doc/​Documentation/​vfio.txt]] for the details and VFIO. +VM1, VM2 correspond ​to the examples in his proposal. VM1 can express access permission (R/W) to its guest physical address (GPA) regions ​by virtual IOMMU. Typically, IOMMU (e.g.  AMD-Vi and Intel VT-d) uses page tables that convert bus (I/O) address to GPA for a given PCI device to protect the rest of the system from the DMA operations ​when specifying the DMA addresses. See [[https://​www.kernel.org/​doc/​Documentation/​vfio.txt]] for the details and VFIO. 
  
 If you think about an imaginary (i.e. virtual) PCI device "​**R**",​ you can set up mapping from bus (I/O) address to GPA for that device (because you can set up such mapping for each PCI device). This way, VM1 "gets full control of its security, from mapping all memory (like with current vhost-user) to only mapping buffers used for networking (like ivshmem) to transient mappings for the duration of data transfer only." If you think about an imaginary (i.e. virtual) PCI device "​**R**",​ you can set up mapping from bus (I/O) address to GPA for that device (because you can set up such mapping for each PCI device). This way, VM1 "gets full control of its security, from mapping all memory (like with current vhost-user) to only mapping buffers used for networking (like ivshmem) to transient mappings for the duration of data transfer only."
Line 13: Line 13:
  
  
-If the device driver of **R** sets up a buffer to receive dataprogramming the registers, data will be put into the buffer as DMA. Then the device R generates an interrupt (maybe putting more data to other buffers). The IOMMU makes sure that data (addressed by bus or I/O address) be translated to GPA. If the mapping is not valid, then IOMMU reports an error (via MSI). This way, VM1 is protected against the DMA operations made by the device R. Once the DMA operation is done, the IOMMU transaction is done.+If the device driver of **R** sets up a buffer to receive data by programming the (virtual) ​registers ​of **R**, data will be placed ​into the buffer as DMA. Then the device R generates an interrupt (maybe putting more data to other buffers). The IOMMU machinery ​makes sure that data (addressed by bus or I/O address) be translated to GPA. If the mapping is not valid, then IOMMU reports an error (via MSI). This way, VM1 is protected against the DMA operations made by the device R. Once the DMA operation is done, the IOMMU transaction is done. Data transferring from VM1 basically takes the same steps.
  
 ==== Inter-VM Communication ==== ==== Inter-VM Communication ====
  
-How can this mechanism ​is used for inter-VM communication?​ It should be easyIn this exampleVM1 receives data from VM2. For example, DPDK runs on VM2, forwarding packets to VM1.+How is this mechanism used for inter-VM communication?​ It should be straightforwardTake a look at a simple ​example ​where VM1 receives data from VM2. For example, DPDK runs in VM2, forwarding packets to VM1.
  
-One of the simplest ways would be (Rx for VM1): +In VM1 the device driver uses polling to keep DMA operations of **R** open, looking at the "**bus address**"​ (the step numbers below should match with the one in the figure): 
-  - In VM1 the device driver uses polling to keep DMA operations of **R** open +  - For performance reasons, the buffer addresses ​would be static or covered by larger regions that are mapped by virtual IOMMU. The mapping is determined and established by VM1
-  - For performance reasons, the buffer addresses ​should ​be static or covered by larger regions that are mapped by virtual IOMMU. The mapping is determined and established by VM1. +  - QEMU of VM1 communicates the configuration of virtual IOMMU to QEMU of VM2. This would require extensions to the vhost-user protocol. 
-  - A process or kernel in VM2 accesses BAR + (bus address) in its GPA to copy data to VM1 +  - The vhost-pci (implemented by the extension) sets BAR in VM2.  
-  - +  - A process ​(e.g. DPDK) or kernel in VM2 accesses BAR + (**bus address**) in its GPA to copy data to VM1. This operation can be done by data-copying or DMA (by SR-IOV VFs, for example). 
 +  - The mapping from the **bus address** to GPA in VM1 is done by virtual IOMMU configured for the device R. 
  
  
 {{:​vm2vm_mst.png|}} {{:​vm2vm_mst.png|}}
vm2vm_mst.txt · Last modified: 2015/09/03 04:33 by Jun Nakajima