|
The basic virtual memory-mapped communication (VMMC) model provides protected, direct communication between the sender's and receiver's virtual address spaces, but it does not support connection-oriented high-level communication APIs well. VMMC-2 is an extension to the basic VMMC. We describe the design, implementation, and experiments of three mechanisms in the VMMC-2: a user-managed TLB mechanism for address translation which enables user libraries to dynamically manage the amount of pinned space and requires only driver support from many operating systems, a transfer redirection mechanism which allows to avoid a copy on the receiver's side, and a reliable communication protocol at the data link layer which allows to avoid a copy on the sender's side.
To demonstrate these techniques and their integration, we have designed an extended virtual memory-mapped communication model VMMC-2 and implemented it on PCs connected with Myrinet. This is a completely new implementation, not an incremental improvement of VMMC. We have also implemented a stream sockets protocol on top of VMMC-2 to demonstrate how to take advantage of the new features. Our micro-benchmarks show that the one-way latency of VMMC-2 is 13.5 microseconds with transfer redirection and retransmission and the maximum bandwidth is over 90 MBytes/s. The one-way latency of the stream sockets layer is 20 microseconds and its bandwidth is over 84 MBytes/s. We experimented with several applications using the stream socket facility and showed that these techniques are indeed effective. |
